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Abstract 

This  study  focused  on  improving  viglance  performance  through  developing  methods  to 
arouse  subjects  to  the  possibility  of  errors  in  a  data  manipulation  information  warfare  attack. 

The  study  suggests  that  by  continuously  applying  arousal  stimuli,  subjects  would  retain  initially 
high  vigilance  levels  thereby  avoiding  the  vigilance  decrement  phenomenon  and  improving  error 
detection.  The  research  focused  on  which  methods  were  the  most  effective  as  well  the  impact  of 
age  upon  the  arousability  of  the  subjects.  Further  the  implications  of  vigilance  and  vigilance 
decrement  for  correct  detections  as  wpll  as  productivity  were  explored. 

The  study  used  a  simulation  experiment  to  provide  a  yigilance  task  in  an  reality-based 
information  warfare  environment.  The  results  of  the  study  suggest  that  stimuli  type  and  age  do 
impact  arousal,  although  stimuli  type  had  the  greater  effect.  Also,  moderate  support  was  found 
to  indicate  that  arousal  does  affect  vigilance  and  vigilance  decrement.  However,  the  final 
analysis  revealed  that  it  was  the  arousal-vigilance  interaction  that  had  the  greatest  impact  on 
cprrect  detection  and  productivity. 
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THE  AFFECT  OF  VARYING  AROUSAL  METHODS  UI)ON  VIGILANCE  AND  ERROR 
DETECTION  IN  AN  AUTOMATED  COMMAND  ANP  CONTROL  ENVIRONMENT 


CHAPTER  1:  INTRODUCTION 


Background 

“Throughout  history,  military  leaders  have  regarde4  information  superiority  as  a  key 
enabler  of  victory”  (JV  2020  2000:  8).  The  situation  has  changed  little  over  time.  Today’s 
modem  military  relies  heavily  upon  the  instantaneous  transmissjon  of  volumes  of  information  to 
coordinate  battlefield  actions.  Weapon  systems  such  as  global  positioning  satellites,  cruise 
missiles,  and  heads-up-displays  in  cockpits  are  some  examples  pf  military  technologies  that  rely 
upon  the  ability  to  maintain  accurate  information  flow.  The  importance  of  information  extends 
even  further  when  considering  the  critical  nature  of  reliable  communications  between 
commanders  and  the  front  lines.  To  emphasis  the  importance  of  information  to  the  military. 
Joint  Vision  2020  states  that,  “the  joint  force  must  be  able  to  take  advantage  of  superior 
information  converted  to  superior  knowledge  to  achieye  decision  superiority”  (2000: 9). 

To  combat  the  advantages  afforded  by  information  supremacy,  a  new  type  of  warfare  has 
evolved  called  information  warfare  (IW).  In  its  purest  form,  IV/  is  simply  any  aspect  of  warfare 
related  to  gathering,  manipulating,  or  denying  information  (Schwartau  1994).  In  this  sense,  IW 
is  not  an  entirely  new  concept.  Its  foundations  can  be  traced  to  ancient  times  when  a  military 
commander  would  use  scouts  or  spies  to  provide  critical  information  concerning  an  adversary’s 
capabilities  or  location.  In  addition  to  information  gathering,  disinformation  has  been  widely 
used  as  an  IW  tactic.  In  World  War  II,  German  commanders  were  led  to  believe  the  attack  on 
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Normandy  Beach  would  actually  have  happened  further  east  at  Calais  (Weinberg  1994).  In 
today’s  context,  IW  has  been  extended  to  include  the  vast  network  of  computers  we  commonly 
call  the  Internet.  While  immensely  powerful  and  widely  used  by  the  military,  the  Internet  is 
vulnerable  to  an  IW  attack.  In  many  ways,  IW  has  become  the  “poor  man’s”  weapon  of  choice 
as  a  complex  IW  attack  can  be  staged  from  a  single  computer  located  nearly  anywhere  in  the 
world  (Schwartau  1994). 

Statement  of  Problem 

While  IW  can  take  many  forms,  of  particular  interest  to  this  study  is  the  practice  of 
manipulating  the  data  the  enemy  uses  for  decision  making.  Data  manipulation  can  occur  when 
an  adversary  gains  access  to  computers  or  databases  maintained  by  the  military.  In  this  case,  the 
intruder’s  intent  is  not  to  deny  service  or  cause  disruption.  Rather,  it  is  his  or  her  intent  to 
manipulate  or  modify  the  data  to  cause  the  system’s  user  to  make  erroneous  decisions  based 
upon  the  incorrect  data  (Bizante,  Llinas,  Seong,  Finger,  and  Lian  2000).  For  example,  if  a 
commander  is  basing  his  attack  decisions  upon  a  reconnaissance  report  stating  the  enemy  is  just 
over  the  next  ridge  and  that  data  has  been  modified  to  show  that  the  enemy  is  elsewhere,  the 
attack  will  fail  and  jives  and  resources  could  be  lost. 

The  current  dependence  of  the  U.S.  military  on  information  necessitates  the  prompt 
detection  of  any  altered  data.  As  a  result,  wide  arrays  of  hardware  and  software  devices  have 
been  created  to  detect  and  prevent  IW  attacks.  Despite  the  advanced  technology  in  place  today, 
in  many  cases  it  falls  to  the  users  of  the  data  to  judge  the  validity  of  the  data  provided.  Therefore 
the  users  must  be  attuned  to  the  possibility  that  an  IW  attack  wa^  successful  and  the  data  they  are 
examining  may  be  erroneous.  Vigilance,  as  defined  by  Mackworth  (1957)  is,  “a  state  of 
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readiness  to  detect  and  respond  to  certain  changes  occurring  at  random  times  intervals  in  the 
environment”.  User  vigilance  then  may  be  the  key  to  immediate  detection  of  data  manipulation 
errors.  Unfortunately,  extensive  studies  have  demonstrated  that  over  time,  vigilance  will 
decrease  due  to  a  variety  of  causes  (Davies  &  Time  1969;  Poulton  1977;  Singh  &  Malloy  1993). 
Such  an  occurrence  has  been  labeled  “vigilance  decrement1”  (Parasuraman  1984:  250).  The 
ramifications  of  such  a  defect  for  the  militaiy  are  significant.  Therefore,  a  method  must  be 
developed  to  increase  information  technology  user’s  vigilance  while  limiting  the  vigilance 
decrement  over  time. 

Statement  of  Research 

Real-world  and  laboratory  experiments  have  suggested  that  human  ability  to  continuously 
detect  errors  becomes  less  reliable  over  time  (Davies  &  Parasuraman,  1982;  Balakrishnan  1998). 
Consequently,  substantial  work  has  been  accomplished  to  try  and  determine  what  kinds  of 
factors  affect  vigilance  and  how  the  decrement  can  be  reduced  pr  avoided  altogether  (Singh  and 
Molloy  1993;  Wickens  1999).  One  concern  has  been  the  effect  of  arousal,  or 
psychophysiological  stimulation,  towards  detection  of  signals  (Paus  &  Zatorre  1997).  Arousal 
heightens  an  individual’s  receptiveness  or  sensitivity  toward  some  activity.  In  this  situation 
arousal  heightens  the  operators  vigilance  or  state  of  readiness  tq  detect  an  IW  attack.  According 
to  Parasuraman  and  Davies,  arousal  does  little  to  eliminate  vigilance  decrement  (1984).  Rather 
arousal  affects  the  overall  level  of  vigilance.  Based  on  that  principle,  by  continuously  arousing 
an  individual  to  the  possibility  of  data  manipulation  and  thereby  continuously  maintaining  a  high 
overall  level  of  vigilance,  his  or  her  vigilance  decrement  can  be  theoretically  eliminated  for  a 
period  of  time. 
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In  order  for  arousal  to  be  continually  effective,  methods  must  be  developed  that  can  be 
used  interchangeably  to  maintain  a  high  level  of  vigilance.  Mehrabian  and  Russell  (1974) 
discovered  that  arousal  states  can  “decrease  when  situations  are  simple,  unchanging,  familiar, 
and/or  predictable”.  Further,  it  was  found  that  repeated  exposure  to  the  same  stimulus  would 
decrease  arousal  over  time  (Mehrabian  1995).  Therefoje,  the  methods  of  arousal  would 
seemingly  need  to  be  altered  as  well  as  their  frequency  of  application. 

Research  Objectives 

If  it  were  possible  to  make  an  individual  continuously  sensitized  toward  detecting  errors, 
the  IW  tenant  of  data  manipulation  would  be  greatly  hindered  (Cohen  1994).  The  potential 
benefits  to  the  military’s  information  operations  would  be  dramatic  as  incorrect  or  manipulated 
data  would  be  detected  more  frequently.  Therefore,  this  research  will  explore  the  effects  of 
varying  arousal  techniques  as  a  method  to  increase  an  individual’s  vigilance  or  awareness 
towards  detecting  incorrect/manipulated  data  without  decreasing  the  overall  effectiveness  of  the 
individual’s  decision  making.  Using  a  simulation  experiment,  varied  arousal  methods  will  be 
applied  to  subjects  performing  real-world  tasks.  The  results  of  the  experiment  will  lead  to  the 
development  of  procedures  system  designers  and  information  operations  personnel  could 
implement  throughout  the  Air  Force  to  defend  against  information  warfare  attacks. 

Paper  Overview 

Chapter  two  contains  this  study’s  literature  reviejv,  which  explores  previous  work 
accomplished  in  the  vigilance  and  arousal  fields  and  presents  findings  relevant  to  the  current 
study.  Of  particular  concern  are  the  prior  studies  that  included  arousal  methods  as  part  of  the 
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research,  intended  or  not.  Additionally,  the  literature  will  lend  itself  to  the  development  of 
hypotheses  related  to  the  types  of  arousal  most  effective  in  improving  vigilance  and  error 
detection.  The  methodology  discussed  in  chapter  three  provides  pertinent  information  about  the 
sample  population  chosen  for  the  study  and  describes  the  instrument  that  was  used  to  collect  the 
data.  It  also  discusses  the  design  of  the  experiment  as  well  ^s  procedures  used  to  analyze  the 
data  once  collected.  A  sample  of  the  data  collection  instrument  as  well  as  the  questionnaires 
used  is  provided.  Finally,  the  methodology  chapter  discusses  steps  taken  in  the  actual  data 
collection  process.  The  results  of  the  data  analysis  arp  discussed  in  chapter  four  and  a 
description  of  the  data  is  presented.  The  results  are  then  compared  to  the  hypotheses  developed 
in  chapter  two.  The  comparison  results  are  then  presented  and  discussed.  Chapter  five  goes  on 
to  summarize  the  findings  of  the  study  and  draws  conclusions  op  the  results  of  the  data  analyzed 
in  chapter  four.  Recommendations  for  future  related  research  are  also  provided  in  the  final 
chapter. 
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CHAPTER  2:  LITERATURE  REVIEW 


Introduction 

This  chapter  reviews  literature  relevant  to  the  study  of  vigilance  performance  and 
possible  arousal  methods  that  may  influence  the  error  detection  aspect  of  vigilance.  This  chapter 
begins  with  background  on  how  information  technology  has  led  to  the  development  of 
information  warfare  data  manipulation  tactics.  It  then  proposes  vigilance  as  a  method  for 
defense  and  discusses  its  related  problems.  Arousal  is  introduced  and  its  effect  upon  vigilance  is 
explored.  Finally,  a  theoretical  model  and  related  hypotheses  are  developed  for  the  research 
involved. 

Information  technology  (IT)  in  the  Military 

In  the  modem  age  of  computers,  high-speed  telecommunications,  and  digital  everything, 
the  operational  environment  for  the  militaiy  has  undergone  considerable  change.  Even  on  the 
battlefield,  computer  technology  can  be  readily  seen  either  in  the  form  of  laptops  or  the  less 
obvious  internal  circuitry  for  fly-by- wire  aircraft  or  stateTof-the-art  tanks  (Webster  1998). 
Increasingly  popular  among  commanders  are  automated  decisjon  support  aids.  These  tools  are 
designed  to  bring  together  large  amounts  of  information  and  present  it  in  an  efficient  manner  for 
human  decision  makers  to  understand  (Fox  and  Boehm-Davis;  DeWaard  and  Brookhuis  1998). 
These  technologies  allow  for  tremendous  increases  in  productivity  and  operational  capability, 
however,  they  also  place  a  large  burden  upon  the  informatipn  technology  to  determine  what 
information  is  important  £nd  whether  or  not  it  is  accurate. 

As  pervasive  as  the  information  technology  has  become,  there  seems  an  endless  potential 
for  its  use.  The  Chairman  of  the  Joint  Chiefs  outlines  how  the  Department  of  Defense  envisions 
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information  technology  will  be  used  to  gain  information  superiority  over  any  adversary  in  the 
forward  looking  Joint  Vision  2020.  He  predicts  that,  “Information  superiority  will  enhance  the 
capability  of  the  joint  commander  to  understand  the  situation,  determine  the  effects  desired, 
select  a  course  of  action  and  the  forces  to  execute  it,  accurately  assess  the  effects  of  that  action, 
and  reengage  as  necessary  while  minimizing  collateral  damage.”  (JV  2020  2000:  22).  From  this 
document  it  is  clear  the  military  intends  to  extract  every  advantage  possible  from  the  information 
technology  available. 

Despite  the  many  advantages  afforded  by  information  technology,  computers  alone 
cannot  replace  the  humans  they  are  designed  to  aid  (Alberts  1996).  Instead,  a  divided  role 
emerges  as  computers  are  assigned  tasks  that  are  either  too  difficult  or  monotonous  for  humans 
to  perform.  In  many  situations  the  duties  are  even  shared  as  humans  and  computers  function  as 
joint  monitors  for  critical  processes  or  even  entire  systems  (Para§uraman  1987).  Such  interaction 
requires  the  operators  to  have  a  high  degree  of  understanding  of  the  role  and  limitations  of  the 
computer  (Webster  1998).  The  operator  must  be  able  to  assume  total  control  of  the  system  in  the 
event  of  computer  failure. 

Vulnerabilities  of  IT 

By  some  estimates,  modem  computer  systems  have  reached  reliability  rates  spanning 
from  78%  to  95%  (Jones  2000).  Due  to  these  consistent  reliability  rates,  a  tendency  to  rely  upon 
the  information  technology  to  provide  completely  accurate  and  reliable  outputs  develops. 
Ironically,  preliminary  studies  with  National  Aeronautics  ancj  Space  Administration  astronaut 
crews  have  shown  that  automated  procedural  and  decision  aids  have  the  effect  of  increasing  error 
rather  than  reducing  it  (Mosier  et  al  1995: 991).  Mosier  posit?  “automation  commission  errors 
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are  errors  made  when  crews  do  not  take  appropriate  action  bepause  they  are  not  informed  of  an 
imminent  problem  or  situation  by  automated  aids.”  (1995:  992).  Computer  operators  can  fall 
into  the  trap  of  believing  the  equipment  does  not  make  any  mistakes  and  therefore  neglect  their 
shared  duty  of  monitoring  the  system.  In  the  event  of  a  data  error,  it  is  even  possible  that  the 
operator  would  ignore  it  altogether  because  of  their  false  sense  of  security  in  the  information 
technology. 

Joint  Vision  2020  realizes  that  information  systems,  processes,  and  operations  add  their 
own  sources  of  friction  and  fog  to  the  operational  environment  (2000: 10).  Computers  are  not  all 
user  friendly  and  many  take  a  great  deal  of  experience  <md  knowledge  to  operate.  In  the 
compressed  time  and  space  of  a  battlefield,  any  problems  that  opcur  with  computers  often  cannot 
be  dealt  with  appropriately;  thereby  leading  to  mistakes  in  the  information  they  produce. 
Combine  these  errors  with  our  ever-increasing  dependence  op  information  processes,  systems, 
and  technologies  and  a  potential  vulnerability  arises  that  mipt  be  defended.”  (JV  2020  2000: 

29).  For  the  military  this  vulnerability  is  manifested  in  the  realm  of  information  warfare. 

Information  Warfare  as  a  Weapon 

Information  warfare,  at  least  as  it  applies  to  information  technology,  is  a  new  facet  of  the 
modem  defense  structure.  Joint  Vision  2020  defines  it  as  “those  actions  taken  to  affect  an 
adversary’s  information  and  information  systems  while  defending  one’s  own  information  and 
information  systems  (2000: 26).  In  an  information  warfare  environment,  the  information 
technology  structure  the  military  (and  society)  depends  upon  to  gain  advantage  will  be  the  target 
of  the  attack.  The  attack  methods  are  widely  varied  but  gene/ally  consist  of  efforts  to  disrupt, 
destroy,  or  deny  access  to  information  vital  to  the  enemy’s  operation.  In  the  information  warfare 
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arena,  anyone  with  a  computer  or  any  other  form  of  information  technology  is  vulnerable  to 
attack. 

Information  warfare  is  unlike  conventional  warfare  in  that  it  can  be  waged  from  relatively 
safe  or  remote  distances,  it  costs  little  to  do,  and  almost  anyone  (not  just  a  nation-state)  can 
conduct  it.  The  increased  availability  of  the  public  internet,  pommercial  satellites,  and  digital 
communications  all  give  potential  adversaries  relatively  low  cost  weapons  and  associated 
capabilities  (JV  2020  2000:  4).  Furthermore,  these  potential  enemies  are  likely  to  be 
nontraditional  in  nature  and  their  application  of  war.  They  will  often  be  small  groups  with 
limited  objectives,  yet  the  damage  they  could  potentially  infljct  would  rival  that  of  the  largest 
land  armies.  In  a  1995  the  National  Defense  University  speculated  that,  while  unlikely,  an 
organized  information  warfare  attack  focused  on  the  telecommunications,  financial,  and 
industrial  networks  could  bring  this  country  “to  its  knees”  faster  than  a  armed  invasion  short  of 
nuclear  war  (Libicki  1995:  60). 

Data  Manipulation 

t 

While  information  warfare  can  assume  many  forms,  of particular  concern  to  this  study  is 
the  practice  of  data  manipulation.  McComack’s  Information  Manipulation  Theory  (IMT) 
provides  critical  background  on  the  concept  of  deception  jn  communications  (Biros  1998). 
Essentially,  McCornack  argues  that  the  nature  of  deceptiop  falls  within  how  information  is 
manipulated  within  each  message.  In  a  data  manipulation  environment  consistent  with 
information  warfare,  the  method  of  deceiving  the  enemy  is  tljrough  distorting  the  information 
that  is  received  by  the  enemy  in  order  to  lead  him  or  her  into  poor  decision-making  (McCornack 
1992).  In  congruence  with  IMT,  potential  adversaries  could  launch  information  attacks  with  the 


9 


intent  of  corrupting  data  sources  as  well  as  the  data  enroute  through  the  information  systems 
(Bisantz  et  al  2000).  These  attacks  would  be  designed  to  deceive  any  decision  makers  relying 
upon  the  information  by  altering  in  some  fashion  the  content  of  the  information.  “An  attacker 
allows  the  target  to  continue  to  operate  but  manipulates  the  information  the  target  collects, 
generates,  carries,  or  analyzes”  (Webster  et  al  1998:  1 1).  The  decision  maker,  dependant  upon 
the  automated  decision  aid  in  place,  could  make  decisions  that  are  potentially  fatal  to  his  or  her 
personnel,  or  at  theyery  least,  give  aid  to  the  enemy. 

Unfortunately,  data  manipulation  attacks  are  difficult  tojdetect  and  are  usually  conducted 
without  physically  compromising  the  targeted  system  (Llinas,  Bialas,  and  Chen  1998). 

Undetected  IW  techniques  could  be  used  to  distort  information  provided  to  the  enemy  by  their 
own  information  and  decision  aiding  systems  thereby  disrupting  their  trust  in  or  probable  use  of 
such  systems.  Alternatively,  however,  one  might  want  to  intentionally  deceive  an  enemy  with 
the  purpose  of  having  him  continue  to  trust  the  altered  infomjation  (Bisantz  et  al  2000).  If  the 
enemy  engaging  in  data  manipulation  attacks  opted  for  the  second  option,  significant  damage 
could  be  obtained  by  convincing  the  decision  maker  that  the  information  he  or  she  is  using  is 
correct.  The  attacker  could  effectively  steer  the  operator  into  a  disastrous  situation.  The  covert 
methods  the  attacker  uses  may  allow  for  repeated  manipulation  without  detection.  In  the  course 
of  a  long  campaign,  such  actions  could  be  pivotal  inyhe  final  outcome  of  the  conflict. 

Vigilance  as  a  Counter  to  IW 

Due  to  the  subliminal  nature  of  data  manipulation  attacks,  the  role  of  the  human  operator 
becomes  greatly  elevated.  Once  the  information  has  been  altered,  the  decision  maker  stands  as 
the  fast  line  of  defense  to  avert  the  attack  if  automated  detection  systems  fail  (Alberts  1996).  The 
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question  then  becomes,  what  can  the  operator  do  to  detect  and  alert  the  decision  maker,  if 
different  from  the  operator,  to  potentially  erroneous  data?  That  answer  to  that  question  may  lie 
within  the  field  of  vigilance  research. 

N.  H.  Mackworth  defined  vigilance  as  “a  state  of  readiness  to  detect  and  respond  to 
certain  small  changes  occurring  at  random  time  intervals  in  the  environment.”  (Parasuraman  and 
Davies  1984:  244).  This  state  of  readiness  implies  a  certain  sustained  attention  or  alertness  to 
specified  stimuli,  possibly  for  prolonged  periods  of  time  (Dittjnar  and  Warm  1993).  Vigilance 
therefore  is  a  state  of  awareness  within  the  operator  to  be  able  to  detect  any  changes  in  the  data 
that  are  not  congruent  with  known  or  expected  outputs.  In  particular,  the  operator  would  be 
looking  for  data  that  does  not  fit  or  is  not  sensible  within  the  context  of  the  operation  being 
carried  out.  A  vigilant  operator  would  focus  his  or  her  attention  on  any  subtle  change  within  the 
system  they  are  responsible  for  and  determine  if  that  change  was  the  result  of  a  data 
manipulatioij  attack  (Paus  and  Zatorre  1997). 

f 

The  effect  vigilance  can  have  on  error  detection  is  critical.  Balakrishnan  noted  that  any 
breakdown  in  vigilance  or  sustained  attention  to  the  information  system  in  use  could  cause  the 
operator  to  overlook  orrespond  too  slowly  to  a  mission-critical  event  (1998).  In  the  case  of  a 
data  manipulation  attack,  a  vigilant  operator  may  be  able  to  detect  the  altered  data  and  alert  the 
decision  maker  before  the  dnemy  is  able  to  gain  an  advantage  from  the  attack.  For  all  the 
intelligence  a  computer  appears  to  have,  once  breached,  it  usually  cannot  tell  a  valid  input  from 
an  invalid  one  entered  by  an  attacker.  If  the  system  software  i§  unable  to  determine  which  data 
has  been  altered,  it  will  fall  solely  to  tjie  operator  to  detect  any  change. 
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Vigilance  Decrement 

While  vigilance  may  be  the  key  to  protecting  against  data  manipulation,  research  has 
shown  that  humans  are  not  adept  at  sustaining  attention  fpr  long  periods  of  time  (Sing  and 
Molloy  1993;  Wickens  1999).  Mackworth,  in  1950,  first  labeled  the  inability  of  an  observer  to 
correctly  detect  infrequently  presented  signals  over  time  as  the  “vigilance  decrement”  (Weiner 
1987:  729).  Further  research  by  Prarasuraman  and  Davies  confirmed  this  finding  (1984).  In 
general,  the  longer  the  observer  remains  in  a  vigilance  type  task  the  greater  the  decrease  in 
detection  accuracy  and  speed.  While  empirical  proof  exists  that  a  vigilance  decrement  can  occur, 

the  exact  causes  arp  widely  debated  (Weiner  1987). 

\ 

Parasuraman  and  Davies  argue  that  the  vigilance  decrement  and  level  of  vigilance  are 
separate  phenomenon  and  may  be  caused  by  different  processes  (1984).  In  past  research,  a 
vigilance  decrement  could  be  found  in  a  variety  of  situational  tasks  (Weiner  1987;  Helton, 
Dember,  Warm,  and  Mathews  2000).  It  would  seem  that  regajdless  of  task  type,  if  vigilance  is 
required  a  decrement  could  and  often  does  occur.  The  question  then  becomes,  what  was  the 
initial  level  of  vigilance  at  the  beginning  of  the  task?  According  to  the  above  argument  by 
Parasuraman  and  Davies,  the  initial  level  of  vigilance  may  havq  been  lower  in  some  of  the  tasks 
performed.  A  lower  initrallevel  could  cause  a  perceptually  greater  decrement  at  least  in  terms  of 
percentage  or  critical  signals  missed.  Oppositely,  a  higher  level  of  initial  vigilance  may  in  fact 
experience  the  decrement  but  appear  to  be  less  because,  at  leas)  initially,  the  subject  was  able  to 
detect  more  signals  than  the  person  would  had  a  low  vigilance  level  to  begin  with. 

There  are  a  variety  of  possible  factors  that  contribute  to  the  initial  level  of  vigilance 
within  the  computer  operator.  Such  factors  could  include  physical  or  psychological  conditions 
within  the  subject,  environmental  influences,  or  even  varied  diurnal  constraints  that  interfere 
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with  the  ability  or  desire  to  accomplish  the  given  task  (Paraspraman  and  Davies  1984).  These 
problems  have  been  widely  studied  and  remedies  are  available  to  reduce  their  impact  on 
performance  (Dittmar  and  Warm  1993).  Even  without  these  conditions,  the  vigilance  decrement 
can  still  exist.  One  possible  explanation  for  the  phenomenon  is  complacency.  Having  been  the 
subject  of  a  considerable  portion  of  past  vigilance  studies,  the  Aviation  Safety  Reporting  System 
developed  their  definition  of  complacency  as  a  “self-satisfaction,  which  may  result  in  non¬ 
vigilance  due  to  an  unjustified  assumption  of  satisfactory  systepi  state  and  performance”  (Singh 
and  Molloy  1993:  357).  Wickens  further  expands  the  definition  as  any  situation  in  which 
operators  are  not  forced  to  consider  the  nature  of  the  decision  they  are  making  because  it  has 
been  automated  and  their  attention  has  been  diverted  to  other  (asks  (1999:  6).  In  an  automated 
environment,  there  is  great  temptation  to  allow  the  computers  tq  do  all  of  the  work,  especially  if 
the  task  is  complex  and  difficult  to  understand.  In  these  situations,  operators  can  become 
complacent  in  their  monitoring  of  the  system  and  consequently  reduce  their  initial  levels  of 
vigilance  at  the  onset  of  the  task.  In  an  information  warfare  data  manipulation  attack 
environment,  such  reductions  in  vigilance  could  be  devastating.  Methods  need  to  be  developed 
to  ensure  vigilance  performance  does  not  decrease  over  time. 

yigilance/Arousal  Relationship 

If  vigilance  is  to  be  the  key  to  detecting  data  manipulation,  the  problems  of  establishing 
initially  high  levels,  vigilance  decrement,  and  complacency  must  be  eliminated  or  reduced.  One 
possible  way  to  improve  vigilance  is  through  arousal.  Paus  and  Zatore  define  arousal  as  “the 
psychophysiological  condition  that  increases  alertness”  (1977:  392).  In  this  context,  arousal  then 
can  be  any  stimulus  that  elicits  a  psychophysiological  response  in  the  subject  that  increases  his  or 
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her  state  of  awareness.  By  arousing  subjects,  a  method  may  be  developed  to  enhance  their 
performance  in  a  vigilance  task. 

In  1984,  Parasuraman  and  Davies  theorized  that  temporal  changes  in  performance  are 
directly  influenced  by  a  decline  in  ‘perceptual  vigilance’  oj  arousal.  Their  study  found  that 
changes  in  the  autonomic  and  central  nervous  system  activity  during  the  performance  of  the 
vigilance  tasks  showed  that  the  vigilance  decrement  was  accompanied  by  a  decrement  in 
physiological  arousal  over  time  (252).  Verifying  this  theory.  Sing  and  Molloy  discovered  that 
highly  aroused  subjects  had  higher  detection  rates  than  did  lpw  arousal  subjects  (1993:  364). 
Together,  these  findings  suggest  a  strong  correlation  between  arousal  and  vigilance  within  the 
controlled  experiment. 

Need  lj>r  Arousal-Vigilance  Research 

Parasuraman  provided  the  first  empirical  evidence  that  in  a  controlled  setting  where 
subjects  were  responsible  for  providing  back-up  readings  for  an  automated  engine  status 
machine,  poor  monitoring  resulted  from  the  over-reliance  on  automated  decision  aid  (Molloy  and 
Parasuraman  1996).  This  multitask  vs.  single  task  study,  however,  was  criticized  for  not  being 
realistic  due  to  the  artificially  high  failure  rates  in  the  engine  status  system.  In  1996, 
Parasuraman  and  Molloy  modified  the  original  study  to  provide  a  more  real-world  scenario  in 
which  they  were  able  to  show  that  vigilance  does  decline  over  a  20-minute  period  of  time  with 
fewer  failures  present  (1996).  These  findings  uphold  the  theory  of  the  vigilance  decrement  as 
well  as  raise  the  issue  of  how  to  improve  yigilance  levels  in  future  studies. 

Based  upon  the  correlation  of  vigilance  and  arousal,  further  research  is  required  to 
determine  what  kind  of  arousal  is  most  effective  in  sustaining  vigilance  through  the  duration  of  a 
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vigilance  task.  Paus  and  Zatorre  assert  that  the  cognitive  processes  underlying  the  detection  of 
target  stimuli  may  fail  if  the  subject’s  arousal  level  decreases  below  a  certain  point  (1997).  If 
true,  the  stimulus  applicable  to  improving  the  cognitive  function  required  for  error  detection  may 
fall  within  a  cognitive  style  stimulus  (i.e.,  non-physical).  Past  research  has  shown  that  a 
nonspecific  arousal  (alerting)  process  is  preferred  because  of  the  speeding  of  reaction  time 
caused  by  a  neutral  warning  signal  or  accessory  stimulus  (Hackley  and  Valle-Inclan  1999). 

These  two  studies  seem  to  indicate  that  the  method  of  aroqsal  most  effective  for  improving 
vigilance  performance  is  one  that  is  generic  enough  to  stimulate  the  cognitive  function  within  the 
subject.  Further  research  is  required  to  formulate  which  kind  of  arousal  mechanism  would 
produce  the  desired  results. 

In  finding  arousal’s  effect  on  vigilance,  Parasuraman  ^nd  Davies  noted  that  changes  in 
arousal  had  a  greater  effect  on  the  level  of  vigilance  than  on  the  reducing  or  preventing  vigilance 
decrement  (1984).  Paus  and  Zatorre  further  state  that  the  repetitive  nature  of  vigilance  tasks 
requires  that  adequate  levels  of  arousal  be  produced  (1997).  Together,  these  findings  suggest 
that  in  addition  to  nonspecific  cognitive  arousal  stimulus,  the  arpusal  method  must  be  directed  at 
instilling  a  high  initial  level  of  vigilance  and  must  be  repeated  ojten  enough  to  maintain  the  level 
of  vigilance  required  for  the  task.  This  specific  type  of  aipusal  method  requires  additional 
research  in  its  development  and  implementation. 

It  may  be  possible  to  continually  arouse  subjects  to  the  jpint  where  their  initial  levels  of 
vigilance  are  restored  before  the  vigilance  decrement  or  complacency  phenomenon  have  an 
impact.  Based  on  the  arousal  vigilance  research  presented  abovp.  Figure  2-1  was  derived  to  give 
a  theoretical  illustration  of  how  arousal  could  produce  a  consistently  higher  level  of  vigilance. 
Continual  treatments  of  arousal  stimuli  could  return  initial  levels  of  vigilance  to  their  originally 
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high  state,  thereby  reducing,  but  not  entirely  eliminating,  the  overall  vigilance  decrement  effect. 
Line  A  represents  the  vigilance  level  of  the  subject  who  has  received  two  arousal  stimuli,  which 
returned  the  subject  to  his  or  her  initial  high  level  of  vigilance.  In  contrast,  line  B  represents  the 
subject  who  did  not  receive  any  arousal  stimuli  and  continued  to  experience  a  vigilance 
decrement.  Over  time,  the  subject  receiving  arousal  stimulus  would  have  an  overall  higher  level 
of  vigilance  than  the  subject  who  did  not  receive  any  stimulus.  This  graph,  therefore,  illustrates 
how  arousal  may  be  able  to  offset  the  effects  of  deteriorating  vigilance. 


Vigilance 

Level 


Figure  2-1  Theoretical  model  ofthp  Effect  of  Arousal  on  Vigilance 

Arousal  Methods 

One  proven  way  of  arousing  subjects  is  through  the  us$  of  cues,  or  hints  that  something 
may  be  changed  within  the  environment  under  study.  Extensive  research  has  been  conducted  on 
the  application  of  cues  to  help  subjects  improve  error  detection  in  a  controlled  setting  (Tipples 
and  Sharma  2000;  Blocher  et  al  1986).  Davies  and  Tune  proyided  results  showing  that  use  of 
reference  cues  significantly  improved  performance  when  compared  to  the  control  condition  of  no 
reference  cues  at  all  (1969).  Dittmar  reported  similar  findings  when  his  subjects  claimed  that 
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they  had  used  cues  as  an  aid  to  detection  of  critical  signals  (1993).  Finally,  Bisantz  reported  in 
her  studies  of  cue  utilization,  when  cues  were  present  they  greatly  contributed  to  judgment 
performance  (2000).  In  all  of  these  studies  cues  were  given  to  signify  the  potential  for  a  critical 
signal  to  be  present  and  included  a  frame  of  reference  of  what  the  signal  might  look  like. 

Real  world  tests  have  verified  that  using  cues  in  an  operational  setting  can  have  an  impact 
on  the  critical  signal  detection  rate  of  the  test  subjects.  In  1^77,  Murrell  began  a  preliminary 
investigation  into  the  effects  of  simulated  computer  cues  on  a  sustained  visual  monitoring  task. 
His  studies  showed  that  subjects  with  computer  assistance  (cues)  detected  more  signals  than 
those  without  computer  assistance  but  also  made  more  false  alarms  (Parasuraman  1987). 
Similarly,  Yeh,  Wickens,  and  Seagall  examined  an  attention  cueing  tool  for  Army  threat  target 
recognition.  When  such  a  cue  was  present,  subjects  “clearly  detected  the  cued  targets  with 
greater  facility  than  when  cueing  was  not  available”  (Wickens  et  al  1999:  5). 

Unfortunately,  in  an  information  warfare  environment  it  would  often  be  impossible  to  cue 
the  computer  operator  to  the  exact  nature  of  the  data  manipulation.  As  noted  earlier,  data 
manipulation  attacks  are  usually  attempted  in  such  a  way  as  tp  hide  the  manipulation  from  the 
operator.  Therefore,  a  more  realistic  scenario  would  occur  if  a  network  administrator  detected  an 
information  warfare  attack  and  then  issuing  non-specific  warning  to  cue  the  operator  for 
potential  data  manipulation.  Once  aroused  by  the  cue,  the  operator  would  be  more  likely  to 
detect  any  manipulation  that  had  occurred.  Such  a  situation  would  remain  consistent  with  the 
findings  of  Methot  and  Huitema  who  demonstrated  that  artificial  injection  of  signals  to  generally 
identify  potential  targets  can  be  effective  in  producing  high  detection  levels  (1998).  While  there 
are  many  ways  of  providing  cues  to  the  computer  operator,  this  thesis  will  examine  three  types: 
visual,  audio,  £pd  an  audio-visual  combination. 
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Visual  Arousal  Cues 

Visual  arousal  cues  are  those  that  involve  providing  spme  type  of  visual  stimuli  to  the 
user.  Past  studies  have  included  flashing  lights,  pictures,  varied  color  patterns,  or  textual 
references  (Blocher  et  al  1986).  In  2000,  Tipples  and  Sharing  found  that  rapid  flashes  of  light 
across  potential  targets  led  to  improved  target-processing  reaction  times  whether  the  subject 
intended  to  ignore  the  light  or  not.  Their  studies  concluded  that  arousal  is  an  important  factor  in 
maintaining  visual  attention  or  vigilance.  The  arousal  effect  of  visual  cues  is  created  within 
neurological  framework  of  the  human  body.  Davies  and  Tune  jealized  that  in  the  case  of  visual 
stimuli,  increases  in  the  duration  and  intensity  of  the  stimulus  would  produce  higher  detection 
rates  because  of  “temporal  summation  at  the  retina”  (1969: 53).  Given  the  previous  research  and 
the  involuntary  nature  of  the  neurological  system,  any  visual  aropsal  stimuli  cueing  the  subject  to 
the  potential  error  may  yield  excellent  benefits  for  vigilance  tasks. 

In  an  information  technology  based  context,  visual  aroujsal  can  be  easily  conveyed  to  the 
user  through  the  use  of  a  graphical  user  interface.  When  seeking  to  defend  against  data 
manipulation  attacks,  visual  cues  appearing  on  the  computer  monitor  could  be  used  to  alert  the 
operator  to  potential  data  errors.  In  all  of  the  visual  arousal  studies,  the  only  problem  associated 
with  visual  stimulus  is  the  increased  likelihood  of  false  detections  caused  by  over-arousal 
(Tipples  and  Sharma  2000).  This  phenomenon  will  be  discussed  in  a  later  section.  The  next 
section,  however,  explores  another  sepsory  type  of  cue:  audio  signals. 

Audio  Arousal  Cues 

Similar  to  visual  arousal  cues,  the  effectiveness  of  audio  arousal  cues  is  based  upon 
neurological  foundations  (Mehiabian  1995).  These  cues,  however,  are  somewhat  more  limited 


18 


in  application  as  they  generally  take  the  form  of  varied  sounds  9r  tones.  Empirical  evidence  has 
shown  that  the  effects  of  audio  signal  intensity  and  vigilance  performance  are  comparable  with 
the  results  of  studies  employing  visual  arousal  cues  (Davies  and  Tune  1969).  Further,  Dwivedi 
provided  evidence  that  activation  of  audio  arousal  improved  the  detection  of  “high  dominant 
items”  in  his  study  of  how  audio  signals  could  be  used  to  help  single  out  previously  identified 
items  among  a  host  of  distracters.  (Dwivedi  1992:  266).  The  “high  dominant  items”  were  the 
target  items  that  needed  to  be  detected  amongst  other  intermixed  data.  The  presence  of  an  audio 
arousal  cue  improved  the  subject’s  ability  to  detect  the  target  items  (267).  These  studies  suggest 
a  possible  positive  relationship  between  audio  signals  intended  to  arouse  a  subject  and  the 
subject’s  subsequent  ability  to  discriminate  between  critical  signals.  For  vigilance  tasks, 
providing  arousing  audio  signals  may  provide  ap  initially  high  level  of  vigilance. 

Like  visual  arousal  cues,  audio  stimulus  could  be  easily  presented  to  the  subject  through 
an  information  technology  platform.  Speakers  attached  to  a  computer  or  an  audio  system 
installed  for  the  entire  room  would  provide  the  desired  arousal.  The  only  concern  with  audio 
arousal  would  be  a  training  mechanism  to  orient  the  subject  tg  what  each  sound  or  tone  meant. 
Unlike  visual  arousal,  which  can  be  directed  toward  the  specific  problem,  audio  arousal  would 
need  some  type  of  prior  conditioning  as  to  what  the  sounds  indicate.  If  such  conditioning  were 
absent,  the  subject  may  be  aroused  by  the  audio  stimuli  but  woqld  have  no  direction  in  which  to 
apply  the  increased  alertness. 


Interaction  of  Auditory  and  Visual  Cues 

Based  on  the  results  of  previous  work 


in  visual  and  audio  arousal,  it  is  logical  that  a 


positive  interaction  of  the  two  events  could  occur.  Behayioral  and  neurological  evidence 
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supports  the  assumption  that  an  auditory  stimulus  could  have  a  direct  influence  on  the  sensory 
analyses  of  the  visual  stimulus  (Hackley  and  Valle-Inclanj999).  In  1997,  Paus  and  Zatorre 
discovered  that  over  a  period  of  time,  the  visual  cortex  becomes  less  receptive  to  repeated 
stimuli.  However,  when  performing  a  vigilance  task,  “repetitive  auditory  stimulation  would 
gradually  attenuate  such  an  activation  of  the  locus  coeruleps  neurons  and  in  turn  restore  the 
spontaneous  activity  in  the  visual  cortex  to  the  ‘normal’  level.”  (1997:  397).  In  other  words,  an 
auditory  stimulus  could  neurologically  help  prevent  any  decrease  in  the  effectiveness  of  the 
visual  arousal  method. 

Empirical  evidence  exists  to  support  Paus  and  Zatorre’s  theory.  In  1976,  Lees  and  Sayers 
noticed  in  their  studies  of  the  vigilance  decrement  that  little  or  no  decrement  occurred  when  the 
target  signals  were  accompanied  with  a  loud  audiovisual  alarm  that  was  almost  impossible  to 
ignore  (Parasuraman  1987).  Such  results  lead  to  the  possibility  that  while  audio  and  visual 
arousal  cues  alone  can  improve  error  detection  and  reaction  timps,  an  interaction  of  the  stimulus 
may  provide  the  best  method  forraising  the  initial  levels  of  vigilance  within  a  subject. 

Effectiveness  of  Arousal  Methods 

Evidence  has  shown  that  arousal  can  have  an  e|fect  on  the  vigilance  of  subjects 
(Parasuraman  and  Davies  1984:  253,  Paus  and  Zatorre  19Jp7:  393).  Regardless  of  method, 
researchers  have  recorded  improvements  in  error  detection  apd  alertness  (Blocher  et  al  1986: 
458).  Unfortunately,  arousal  is  not  a  foolproof  tool  to  improve  vigilance.  Every  advantage 
afforded  by  arousing  subjects  comes  with  a  potential  cost  that  must  be  mitigated.  Over-arousal, 
habituation,  and  age  considerations  all  may  affect  or  even  linjjit  the  benefits  of  arousal.  These 
problems  are  discussed  below. 
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Productivity  and  Over-Arousal.  Effective  as  arousal  may  be,  evidence  shows  it  is 
possible  to  over  arouse  the  subject.  Hackly  and  Valle-Inclan  determined  that  increased  arousal 
states  lead  to  improved  vigilance  or  ability  to  detect  subtle  changes  in  the  environment,  but  at  the 
cost  of  increased  errors  (1999).  These  errors  generally  come  in  the  form  of  incorrect  detections 
of  target  signals  and  can  produce  a  marked  decrease  in  the  overall  efficiency  if  the  subject  spends 
too  much  time  on  detecting,  identifying,  and  processing  inaccuracies.  Biros  found  in  his  1998 
study  that  arousal  is  positively  related  to  deception  detection  but  at  the  cost  of  increased  false 
detections  (1998).  Subjects,  when  highly  aroused  by  stimuli,  may  begin  to  make  decisions  based 
on  less  information.  In  one  study,  aroused  subjects  were  more  sensitive  to  increasing  demands 
of  error  detection  and  chose  to  perform  less  verification  of  targets  in  order  to  avoid  the  cost  to 
performance  of  missing  any  key  targets  (Mosier  et  al  1999).  These  studies  indicate  that  arousal 
must  be  used  in  moderation  to  avoid  losses  in  productivity.  However,  if  productivity  is  not  as 
great  a  concern  as  correctly  identifying  all  data  errors,  the  level  of  arousal  may  not  matter. 

Habituation.  A  second  problem  with  arousal  is  the  phenomenon  of  habituation. 

Repeated  exposure  to  the  same  stimulus  will  cause  a  steady  drop  in  arousal  state,  known  as 
habituation  (Mehrabian  1995).  As  with  any  signal,  if  it  is  applied  often  enough,  the  receptivity 
of  the  subject  will  decrease.  Just  as  a  stereo  played  loudly  will  not  seem  as  loud  over  time,  a 
subject  exposed  to  an  audio  or  visual  cue  may  be  able  fo  ignore  the  stimulus  altogether. 
Mehrabrian  found  in  his  research  that  such  a  decrease  in  receptivity  is  usually  temporaiy  in 
nature  depending  upon  the  intensity  of  the  stimulus  presentation  (1995: 6).  Parasuraman  and 
Davies  encountered  similar  results  in  their  research  (1984).  Any  researcher  or  information 
system  developer  intent  upon  improving  vigilance  performance  through  arousal  methods  should 
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guard  against  causing  a  habituation  effect  within  their  subjects  by  either  alternating  arousal 
methods  or  changing  some  aspect  of  the  cueing  sequence. 

Age  Issue.  The  final  problem  associated  with  arousal  and  vigilance  is  the  age  of  the 
subjects.  Mehrabian’s  1995  studies  showed  that  “arousable”  persons  show  larger  amplitudes  of 
arousal  and  slower  habituation  of  arousal  to  baseline  or  resting  levels.  In  contrast,  “unarousable” 
persons  have  smaller  amplitudes  and  faster  habituation  (1995:  5).  In  other  words,  subjects  who 
are  more  susceptible  to  arousal  cues  will  show  higher  levels  of  arousal  and  will  be  slower  to 
experience  the  phenomenon  of  habituation  than  subjects  who  are  less  susceptible  to  arousal  cues. 
Mehrabian  argues  that  the  more  trusting  a  subject  is  with  the  environment  they  are  working  in, 
the  more  likely  they  are  to  block  out  or  ignore  the  noise  around  (hem,  thus  being  less  susceptible 
to  arousal  cues  (1995).  In  the  case  of  detecting  data  manipulation  errors  from  an  information 
warfare  attack,  arousal  cues  are  the  intended  noise  that  could  be  unintentionally  blocked  out  by 
the  computer  operator.  If  the  operator  has  significant  trust  with  the  system,  he  or  she  may  be 
prone  to  ignoring  any  type  of  warning  cues  that  may  be  provided.  Raab  suggests  that  the  only 
thing  required  for  trust  is  ignorance  of  how  the  technology  \yorks  and  older  people  tend  to  be 
more  ignorant  of  information  technology  operations  than  younger  people  (1998).  In  a  study 
exploring  the  use  of  decision  aids  in  automobiles,  it  was  found  that  the  older  subjects  implicitly 
trusted  the  decision  aid  due  to  lack  of  knowledge  of  how  the  system  operated  and  a  profound 
sense  of  authority  the  system  provided  (Fox  and  Boehm-Davis,  no  date).  Given  these  results, 
differences  in  trust  of  technology,  associated  with  age,  could  affect  how  arousable  a  subject  is. 

Waard  and  Brookuis  further  examined  the  issue  of  age  and  technology  and  concluded 
that  while  initially  older  people  are  reluctant  to  use  or  accept  technology,  once  they  have 
experienced  the  automation  they  are  more  likely  to  develop  complete  reliance  or  even 


22 


dependency  on  the  system  even  if  they  do  not  understand  hpw  the  technology  works  (1998). 
Similar  findings  were  obtained  in  a  study  of  automated  driving  pids  and  their  acceptance  among 
different  age  groups.  This  research  concluded  that  middle-aged  and  older  groups  were  more 
tolerant  and  accepting  of  inaccuracies  in  technology,  while  younger  age  groups  had  higher 
expectations  and  tended  to  be  less  tolerant  (Fox  and  Boehm-Davis,  no  date).  These  differences 
in  character  traits  among  age  groups  may  lead  to  differences  in  how  arousal  cues  affect  the 
subjects.  It  is  possible  that  younger  individuals  who  are  more  likely  to  question  the  accuracy  of 
computer  technology  may  become  more  aroused  by  visual  or  audio  warning  cues  than  older, 
individuals  who  are  inclined  to  depend  upon  technology. 

Finally,  empirical  evidence  exists  to  suggest  that  the  older  a  person  is,  the  lower  their 
initial  vigilance  will  likely  be  (Pigeau  and  Angus  1995).  Therefore,  an  experienced  operator  who 
is  very  trusting  of  the  system  in  use,  not  only  may  have  a  lower  initial  level  of  vigilance  due  to 
age  but  may  also  be  less  susceptible  to  the  arousal  effects  of  visual  and  audio  cues.  In  this  case, 
the  computer  operator’s  experience  and  trust  may  make  him  or  her  very  difficult  to  arouse  into 
high  state  of  vigilance. 

Regardless  of  arousal  method,  the  above  research  has  shown  that  steps  must  be  taken  to 
limit  the  effects  of  overarousal,  habituation,  and  age  issues.  Once  these  problems  are  overcome, 
it  stands  to  reason  that  arousal  may  lead  to  an  improvement  in  vigilance,  while  limiting  the 
effects  of  vigilance  decrement.  The  following  section  describes  a  theoretical  model  of  how 
arousal  and  vigilance  together  can  lead  to  improved  error  detection. 
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Model  Introduction 


One  of  the  greatest  criticisms  of  vigilance  research  has  been  that  the  laboratory 
experiments  are  not  directly  applicable  to  operational  environments.  Weiner  criticized  the  recent 
vigilance  research  efforts  by  saying  that,  “Greater  efforts  must  be  made  by  vigilance  researchers 
to  develop  methodologies  for  and  to  seek  access  to  data  from  real-world  systems  in  order  to 
evaluate  questions  that  cannot  be  answered  in  the  laboratory.”  (1987:  734).  Given  that  criticism, 
any  future  vigilance  research  would  be  remiss  if  it  did  not  at  address  its  applicability  to  real 
world  issues.  The  following  theoretical  model  illustrates  how  the  application  of  arousal  cueing 
techniques  can  affect  vigilance  and  error  detection  within  a  real-world  setting. 


Fig.  2-2  Theoretical  model  of  Arousal- Vigilance  Interaction 


The  basis  for  this  model  is  wrapped  in  the  research  presented  above  and  its  development 
represents  the  key  interactions  of  arousal,  vigilance  and  the  surrounding  environment.  Following 
from  left  to  right  age  and  arousal  method  are  the  primary  components  leading  to  the  arousal  of 
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the  subject.  For  this  model,  the  arousal  methods  are  the  cue  signals  (audio,  visual,  and 
audio/visual  combination)  that  would  alert  an  individual  to  the  possibility  of  data  errors.  Age,  in 
relation  to  this  study,  is  simply  how  old  the  subject  is  and  shpuld  be  considered  its  own  entity 
and  factor  in  affecting  arousal.  Together,  these  two  constructs  mitigate  the  level  of  arousal 
within  the  subject. 

The  construct,  arousal,  is  the  next  component  of  the  model.  Arousal,  as  defined 
by  Paus  and  Zatorre,  is  the  psychophysiological  condition  that  increases  alertness  (1997).  For 
this  model,  arousal  represents  the  heightened  state  of  alertness  within  the  subject  that  is  the  result 
of  stimuli.  It  is  arousal  that  is  hoped  to  improve  the  next  construct:  vigilance.  It  is  defined  as  a 
state  of  readiness  to  detect  and  respond  to  certain  small  changes  occurring  at  random  time 
intervals  in  the  environment  (Mackworth  1957).  For  this  model,  vigilance  represents  the 
readiness  and  ability  to  detect  altered  data.  As  with  vigilance,  arousal  is  also  hoped  to  reduce  the 
impact  of  the  vigilance  decrement  phenomenon.  Vigilance  decrement,  by  definition,  is  the 
decline  over  time  in  the  rate  of  correct  detection  of  infrequently  presented  signals  (Parasuraman 
and  Davies  1984).  For  this  model,  vigilance  decrement  refers  to  the  decrease  in  detections  of 
manipulated  data  in  the  periods  between  arousal  stimuli.  While  still  expected  to  occur,  the 
arousal  cueing  is  hoped  to  help  limit  its  effects. 

The  model  then  demonstrates  the  relationship  vigilanpe  and  vigilance  decrement  have 
with  the  final  constructs:  error  detection  and  productivity.  First,  error  detection  is  defined  as  the 
ability  to  distinguish  between  noise  and  signal  (Davies  and  Tune  1969).  For  this  model,  error 
detection  is  the  ability  of  the  subject  to  discern  incorrect  data  from  the  correct  data.  The  inclined 
arrow  leading  from  vigilance  to  error  detection  indicates  that  higher  vigilance  states  improve 
error  detection.  Likewise,  the  inclined  arrow  from  vigilance  decrement  to  error  detection 
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suggests  that  as  the  vigilance  decrement  is  reduced,  error  detection  will  improve.  The  next 
construct,  productivity,  is  the  function  of  the  amount  of  work  accomplished  and  the  efficiency  at 
which  it  was  completed  over  time.  The  arrow  from  vigilapce  to  productivity  is  declined  to 
reflect  the  potential  for  arousal-induced  vigilance  to  cause  a  decrease  in  productivity.  Finally, 
vigilance  decrement  is  directly  linked  to  productivity  by  a  inclined  arrow  to  indicate  that  as 
vigilance  decrement  is  reduced,  the  productivity  of  the  subjects  increases. 

Together  these  constructs  explain  how  vigilance  can  be  theoretically  improved  by 
arousal.  Following  the  model  from  left  to  right,  illustrates  the  dynamic  affect  of  each  construct 
on  the  next.  The  following  section  will  explore  the  hypotheses  alluded  to  by  the  model  and 
discuss  their  implications. 

Hypothesis  Generation 

In  general,  the  model  illustrates  the  sequence  of  events  required  for  arousal  to  affect 
vigilance  and  the  potential  outcomes  of  such  an  interaction.  The  following  hypothesis  drives  that 
theoretical  sequence  of  events. 

The  first  part  of  the  model  deals  with  the  component  that  affects  the  arousal  mechanism 
within  the  subjects.  Based  on  past  research,  the  application  of  visual  and  audio  stimulus  to  the 
subjects  will  increase  the  individual  arousal  levels  within  the  subject.  In  a  study  conducted  by 
Hackly  and  Valle-Inclan,  their  results  showed  that  the  self  reported  levels  of  arousal  were 
significantly  higher  for  those  that  received  an  arousal  signal  than  those  that  did  not  (1999).  It  is 
this  increase  in  arousal  that  will  begin  the  process  of  improving  vigilance  performance. 

However,  while  it  is  believed  that  visual,  audio,  and  combination  stimuli  will  improve  arousal, 
each  stimuli  type  affects  the  psycho-physiological  systems  of  the  subject  differently  (1999). 
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Therefore,  first  three  hypotheses  are  directed  at  exploring  each  stimuli  type  individually  and  are 
as  follows: 

HI  a:  The  arousal  method  of  audio  clue  signals  will  have  a  positive  affect  on  arousal. 

Hlb:  The  arousal  method  of  visual  clue  signals  will  have  a  positive  affect  on  arousal. 

Hlc:  The  arousal  method  of  audio  and  visual  combination  clue  signals  will  have  a 
positive  affect  on  arousal. 

Unfortunately,  arousal  may  not  be  effective  for  everyone.  In  addition  to  the  findings  of 
increased  arousal  after  stimulus  application,  other  studies  have  presented  evidence  that  age  levels 
can  affect  how  aroused  a  subject  gets  (Dittmar  and  Warm  1993;  Pigeau  and  Angus  1995). 
According  to  these  studies,  the  greater  age  a  subject  has  will  tend  to  lead  to  a  decrease  in 
“arousability”.  Therefore,  the  arousal  cue  signals  will  be  less  liltely  to  have  an  effect  upon  older 
decision  makers,  thus  Reading  to  the  second  hypothesis: 

H2:  Older  ages  will  limit  the  arousal  effects  of  computer  generated  cue  signals 

There  is  little  doubt,  based  upon  empirical  evidence,  that  an  aroused  subject  will  be  more 
vigilant  (Mosier  et  al  1994).  However,  only  theory  provides  that  arousal  typically  will  affect 
initial  vigilance  levels  and  do  little  to  help  prevent  the  vigilance  decrement  (Parasuraman  and 
Davies  1984).  The  model  indicates  that  a  relationship  between  arousal  and  vigilance  will  exist. 
Additionally,  regardless  of  how  much  arousal  is  applied  to  a  subject,  prior  research  cautions  that 
only  the  initial  level  of  vigilance  will  be  impacted.  These  results  indicate  that  despite  being 
aroused,  the  subjects  are  still  likely  to  demonstrate  a  slight  vigilance  decrement  over  time. 
However,  when  compared  to  subjects  who  are  not  aroused  at  all,  the  decrement  is  less  because 
the  repeated  application  of  arousal  methods  has  restored  the  initial  vigilance  levels  and 
minimized  the  total  decrement  experienced.  This  relationship  is  what  will  possibly  allow  a 


27 


continued  high  state  of  vigilance  and  a  reduction  in  vigilance  decrement.  Therefore  the  third 
hypothesis  is: 

H3a:  Increased  arousal  has  a  positive  association  with  the  level  of  vigilance. 

H3b:  Rates  of  vigilance  decrement  will  be  lower  with  aroused  subjects  than  with  non- 
aroused  subjects. 

Vigilance  was  defined  above  as  an  ability  of  a  subject  to  notice  subtle  changes  within  the 
environment.  If  subjects  are  either  initially  vigilant  or  made  so  through  arousal,  they  should  be 
able  to  successfully  detect  errors  when  present  due  to  their  ability  to  detect  simple  changes 
within  the  environment  they  are  working  in.  Additionally,  if  the  subject  is  not  experiencing  a 
significant  vigilance  decrement,  he  or  she  are  more  likely  tp  remain  able  to  detect  errors  for 
extended  periods  of  time.  The  relationship  between  higher  states  of  vigilance,  reduced  vigilance 
decrement,  and  error  detectipn  leads  to  the  fourth  hypothesis: 

H4a:  Increased  states  of  vigilance  have  a  positive  dissociation  with  successful  error 
detection. 

H4b:  Improved  vigilance  decrement  has  a  positive  association  with  successful  error 
detection. 

While  arousal  may  be  an  effective  way  to  increase  vigilance,  it  is  often  not  without  cost. 
The  productivity  of  subjects,  in  terms  of  time  spent  on  incorrect  detection,  may  be  inversely 
related  to  the  level  of  arousal-induced  vigilance.  In  other  worcjs,  as  arousal  increases  within  the 
subject,  a  decrease  in  productivity  may  simultaneously  occur.  Improved  vigilance  decrement, 
however,  may  mitigate  the  effects  of  decreased  productivity  by  allowing  the  subjects  to  retain 
their  ability  to  detect  small  changes  in  the  environment  around  them  over  time.  This  relationship 
is  the  basis  of  hypothesis  five: 

H5a:  Increased  states  of  arousal-induced  vigilance  have  a  negative  association  with 
productivity. 
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H5b:  Vigilance  decrement  has  a  positive  association  with  productivity. 


The  hypotheses  generated  above  describe  a  theoretical  relationship  between  the 
constructs.  Chapter  three  will  develop  an  experiment  to  examine  the  validity  of  these 
hypotheses.  Only  through  application  can  it  be  known  if  they  are  accurate. 

Summary 

The  need  to  improve  a  subject’s  vigilance  in  critical  tasks  remains  clear.  Arousal  cueing 
may  provide  the  necessary  means  for  increasing  at  least  the  initial  levels  of  vigilance.  If  issues 
of  over  arousal,  habituation,  and  age  can  be  brought  under  control,  an  effective  method  for 
preventing  data  manipulation  errors  as  well  as  those  of  other  detection  tasks  may  be  developed. 
Past  research  suggests  a  correlation  between  arousal,  vigilance  and  vigilance  decrement, 
therefore  this  study  seeks  to  further  refine  the  field  as  well  as  provide  real-world  system 
developers  key  tools  they  can  use  to  prevent  future  mistakes  caused  by  a  lack  of  vigilance, 
particularly  in  the  case  of  data  manipulation  attacks. 
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CHAPTER  3:  METHODOLOGY 


Overview 

The  purpose  of  this  chapter  is  to  describe  the  methods  and  procedures  used  in  conducting 
research  on  the  effects  of  arousal  on  vigilance.  It  includes  a  description  of  population,  sample 
and  measurement  instruments  used  in  the  study.  Additionally,  data  collection  instruments  and 
proposed  analysis  methods  will  be  discussed. 

Objective 

This  study  was  conducted  to  provide  research  on  the  effects  of  vigilance  on  error 
detection  within  an  information  warfare  environment.  As  stated  earlier,  vigilance  may  only  be 
effective  for  a  limited  time.  Therefore,  this  experiment  design  incorporated  various  arousal 
methods  in  an  attempt  to  improve  vigilance  within  the  context  of  a  data  manipulation  attack. 

Additionally,  in  order  to  obtain  greater  value  from  the  outcome  of  this  research,  attempts 
were  made  to  avoid  known  criticisms  of  previous  vigilance  experiments.  In  1984  Mackie 
concluded  “90%  of  the  vigilance  studies  were  conducted  in  a  way  that  that  would  make  it 
doubtful  that  the  findings  could  generalize  to  civil  or  military  systems”  (Adams  1987: 738).  The 
shortcomings  tended  to  include  a  severe  lack  of  realism  that  may  have  been  prevented  by  the  use 
of  simulators  representing  a  multitask  environment  (Gluckman  and  Warm  1993).  This  research, 
therefore  focused  upon  providing  a  reality-based  vigilance  task  within  a  multitask  framework 
with  a  simulation  as  the  primary  means  of  data  collection. 
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Subjects 

Before  creating  the  vigilance  experiment  and  selecting  the  data  collection  instrument,  the 
issues  of  sample  population  and  sample  size  became  a  concern.  For  this  study,  the  sample 
population  consisted  entirely  of  Air  Force  officers.  Officers  are  the  final  decision  makers  in 
most  military  scenarios  and  therefore  were  appropriate  choices  for  a  study  designed  to  improve 
vigilance  and  error  detection.  The  officers  were  divided  into  two  groups  based  upon  age.  Air 
War  College  students,  senior  officers  from  Lt.  Col  to  Colonel  made  up  the  older  aged  group 
while  AFIT  students  and  the  Aerospace  Basic  Course  cadre,  junior  officers  from  lieutenant  to 
captain,  filled  the  younger  aged  group.  Subjects  were  entiped  to  participate  in  the  study  by 
offering  a  $25  gift  certificate  to  the  officers  club  for  the  subject  who  had  the  highest  overall  score 
during  the  simulation. 

A  power  analysis  was  conducted  to  determine  the  numbpr  of  subjects  the  study  required. 

I 

A  power  analysis  was  completed  and  a  total  sample  size  of  48  was  deemed  appropriate.  This 
sample  size  provided,  at  a  significance  of  .05,  a  power  of  .997.  The  results  of  the  power  analysis 
are  located  in  Appendix  A. 

Instruments 

In  searching  for  a  data  collection  tool,  three  requirements  existed:  realism,  modifiability, 
and  simulation  based.  In  coordination  with  the  Aptima  Corporation,  the  Distributed  Dynamic 
Decision-making  system  (DDD)  was  selected  for  use  in  this  study.  DDD  is  a  simulation-based 
program  with  a  high  degree  of  modifiability,  in  which  subjects  can  be  exposed  to  a  variety  of 
military-related  situations.  The  DDD  instrument  is  self-contained  and  can  be  staged  on  nearly 
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any  computer  platform.  While  Aptima  produced  the  shell  of  the  program,  each  individual 
scenario  had  to  be  scripted  and  coded  into  the  system. 

Using  DDD,  each  subject  was  assigned  the  role  of  an  air  defense  commander  deployed  to 
a  hostile  foreign  country  (Appendix  B).  Reinforcing  the  realism  of  the  study,  a  current  potential 
problem  area  of  the  world  was  selected  for  the  scenario.  Each  subject  was  tasked  to  identify 
aircraft  entering  his  or  her  assigned  airspace  based  upon  information  provided  by  two  distinct 
sources:  radar  and  a  decision  support  aid.  The  subject,  after  identifying  the  aircraft,  had  to 
decide  whether  to  allow  the  aircraft  to  enter  the  airspace  or  to  shoot  it  down.  Figure  3-1  provides 
a  snapshot  of  the  screen  displayed  encountered  by  the  subjects. 

Given  the  difficulty  of  encountering  a  new  program,  $n  extensive  training  session  was 
built  into  this  experiment.  This  training  served  two  purposes.  First,  it  educated  the  subjects  on 
the  use  and  functionality  of  the  DDD  system,  while  its  secondary  purpose  was  to  alert  the 
subjects  to  the  distinct  nature  of  the  two  sources  of  information.  For  this  experiment  it  was 
necessary  to  create  separate  sources  of  information  ffonj  which  the  subject  could  discern 
conflicts  in  data,  or  the  errors  caused  by  a  data  manipulation  attack.  This  was  accomplished  by 
providing  the  subject  their  primary  decision  making  data  through  a  decision  support  system 
which  combined  data  from  multiple  radar  sources  over  a  network  vulnerable  to  information 
warfare  attack,  while  allowing  a  secondary  reference  source  of  d^ta  directly  from  a  specific  radar 
site  immune  to  attack.  Appendix  C  contains  the  training  slides  used  for  this  experiment. 
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Figure  3-1  Sample  Screen-Shot  of  the  DDD  System 

A  second  data  collection  tool  used  in  this  study  was  a  questionnaire.  The  questionnaire 
solicited  demographic  data  as  well  as  information  on  hpw  well  the  subject  received  and 
understood  the  training  on  how  to  use  the  DDD  system.  More  importantly  the  questionnaire  also 
provided  self-reported  arousal  levels  for  each  subject.  Using  a  modified  7-point  Lickert  scale, 
employed  by  Hackley  and  Valle-Inclan  in  1999,  the  subjects  were  provided  a  definition  of 
arousal  and  then  asked  to  rate  how  aroused  they  felt  during  the  simulation  (326).  The  four-part 
questionnaire  is  located  in  Appendix  D. 

Experiment  Design 

Viglance  tasks,  as  stated  earlier,  are  those  that  require  sustained  attention  to  detect  subtle 
changes  within  the  environment.  Extensive  studies  have  been  conducted  using  vigilance  tasks 
and  a  volume  of  information  exists  on  their  methods  and  design.  Based  on  these  studies,  Jerrison 
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(1970)  and  Warm  (1984)  outlined  a  framework  for  building  a  vigilance  task  (Percival  and 
Noonan  1987).  The  four  required  elements  are  displayed  in  Table  3-1.  The  design  of  this 
experiment  was  guided  by  the  four  elements. 


1 

The  task  is  prolonged  and  continuous. 

2 

The  signals  to  be  detected  are  highly  discriminable  to  an  alerted  observer. 

3 

The  signalsare+nfrequentandaperiodic. 

4 

The  monitor’s  response  has  no  effect  on  signal  rate. 

Table  3-1 :  Jerrison  and  Warm’s  Framework  for  Vigilance  Research 


Prolonged  and  Continuous.  The  first  element  contains  two  parts,  prolonged  and 
continuous.  Of  great  concern  in  designing  the  experiment  wps  the  length  of  time  in  which  to 
conduct  it.  Teichner,  in  1974,  noted  that  the  vigilance  decrement  was  typically  complete  within 

i 

35  minutes  after  the  initiation  of  the  vigil  (Hollenbeck  et  al  19?5).  Other  vigilance  researchers 
have  considered  a  20-minute  task  to  be  sufficient  for  producing  a  vigilance  decrement  effect 
(Molloy  and  Parasuraman  1996).  Given  these  findings  and  the  constrained  time  to  complete  this 
study  a  20-minute  vigilance  simulation  for  each  subject  was  deemed  to  be  within  the  framework 
outlined  by  Jerrison  and  Warm. 

Regarding  the  continuous  nature  required  by  Jerrison  and  Warm,  “it  has  been  suggested 
that  a  rate  of  at  least  20  -  24  events/minute  are  needed  to  observe  a  vigilance  decrement” 
(Pigeau  and  Angus  1995:  631).  Using  the  DDD  simulator,  the  number  of  possible  events  per 
minute  was  at  least  25,  depending  upon  the  speed  of  the  subject.  These  events  included  the 
subject’s  actions  of  verifying  tracks,  reporting  discrepancies,  and  destroying  enemy  aircraft. 
Based  on  the  prior  research,  this  experiment  satisfied  the  continuity  requirement. 
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The  signals  to  be  detected  are  highly  discriminable  to  an  alerted  observer.  The  second 
requirement  for  a  vigilance  study  is  to  have  highly  discriminable  signals  for  the  subject  to  detect. 
As  mentioned  earlier,  each  subject  was  provided  two  distinct  data  sources.  The  first,  and 
primary,  source  of  data  was  the  display  screen,  which  graphically  represented  the  fused  data  of 
multiple  radar  sites.  On  this  screen,  aircraft  type  and  location  were  presented  through  the  use  of 
icons  to  the  subject  for  action.  The  secondary  source  of  information  appeared  as  messages  from 
individual  radar  sites.  These  messages  also  included  aircraft  type  and  location.  The  signal  the 
subject  searched  for  was  a  discrepancy  between  the  two  sources  of  data.  The  subjects  were 
trained  that  the  graphical  display  operating  over  a  local  area  network  may  have  been  vulnerable 
to  information  warfare  attacks,  while  the  messages  from  radar  sites  are  transmitted  securely  to 
the  subject.  The  subjects  were  able  to  simultaneously  view  tjie  information  provided  by  both 
sources  and  thereby  discern  any  deviations  in  the  data  sources,  thus  satisfying  the  second 
eleipent  of  the  vigilance  framework. 

Thp  cipnals  are  infrequent  and  aperiodic.  Included  witfyn  the  simulations  events  were  a 
total  of  10  erroneous  targets  out  of  55  possible.  Past  research  on  vigilance  and  error  detection 
included  error  rates  ranging  from  1  possible  error  for  the  entire  study  to  an  error  rate  of  1/3  of  the 
total  possible  targets  (Biros  1998;  Molloy  and  Parasuraman  1996).  Given  the  study  attempted  to 
recreate  an  information  warfare  environment  in  which  data  manipulation  occurred,  the  error  rate 
of  1 8%  was  within  reason.  Additionally,  the  errors  needed  to  be  presented  to  the  subjects  in  an 
aperiodic  fashion  to  prevent  the  subjects  from  discerning  a  pattern  within  the  experiment.  This 
experiment  was  divided  into  six  sections,  transparent  to  the  subject,  in  which  the  number  of 
errors  are  distributed  to  each  section  as  follows  1, 2, 2, 3, 2.  This  distribution  helped  prevent  the 
subjects  from  being  aroused  to  a  specific  pattern  of  errors  thereby  reducing  the  validity  of  the 
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experiment.  The  infrequent  and  aperiodic  design  of  this  test  satisfied  the  third  element  of  a 
vigilance  test. 

The  monitor’s  response  has  no  effect  on  signal  rate.  Finally,  in  order  to  build  a  good 
vigilance  experiment,  the  responses  of  the  monitor  must  have  no  impact  on  signal  rate.  Using 
the  DDD  system,  the  errors  were  pre-programmed  to  occur  at  specific  times  and  intervals  and 
could  not  be  changed  or  modified  by  user  actions.  However,  the  time  at  which  the  subject 
discovered  the  errors  depended  upon  the  speed  and  efficiency  of  the  subject.  Therefore,  each 
error  was  given  a  time  window,  corresponding  to  the  sections  mentioned  above,  in  which  it 
would  appear  and  provide  and  opportunity  for  detection  before  it  was  no  longer  available  to  the 
subject  and  counted  as  a  missed  detection.  In  a  multitask,  data  manipulation  setting  it  was  likely 
the  error  may  occur  minutes,  hours,  or  even  days  before  the  altered  information  is  even  presented 
to  the  user.  Regardless  of  detection  time,  the  detection  evenf  will  not  affect  signal  rate  at  all, 
thus  satisfying  the  final  element  of  a  vigilance  experiment. 

Design  Considerations 

While  the  experiment  design  was  based  on  Jerrison  and  Warm’s  framework,  additional 
features  were  added  to  facilitate  data  collection  and  analysis.  In  order  to  objectively  determine 
when  the  subject  noticed  a  discrepancy  in  the  data,  they  were  required  to  click  on  a  discrepancy 
button  with  the  mouse.  At  the  same  time,  the  subjects  were  forced  to  rate  how  confident  they 
were  in  the  decisions  they  made.  The  confidence,  based  on  a  scale  of  one  to  five,  allows 
additional  information  to  be  gained  about  what  the  subjects  really  intended  or  discerned  about 
the  data  they  are  examining  (Balakrishnan  1998).  The  combination  of  a  yes/no  discrepancy 


36 


button  and  the  confidence  level  rating  helped  ensure  the  subject  really  meant  to  express  there 
was  an  error  instead  of  making  a  mistake  or  simply  not  understanding  the  task. 

Finally,  included  in  the  design  of  the  experiment  was  an  incentive  to  perform  well.  A 
weighted  scoring  incentive  was  provided  to  encourage  a  Realistic  environment  of  cost  and 
rewards.  The  scoring  table  used  by  the  subjects  is  located  in  the  training  slides  in  Appendix  C. 
Prior  vigilance  research  has  shown  that  subjects  tend  to  be  aggressive  when  either  costs  are  low 
or  values  are  high  but  conservative  when  the  converse  is  true  (See  et  al  1997: 16).  The  scoring 
system  used  was  designed  to  give  greater  rewards  for  correct  decisions  and  assess  higher  costs 
for  incorrect  decisions  as  their  confidence  rating  increased.  T|iis  system  forced  the  subjects  to 
consider  the  impact  of  the  choices  they  make  in  the  simulation,  similar  to  the  costs  and  rewards 
associated  with  an  actual  military  situation.  Additionally,  the  weighted  scores  helped  avoid  the 
over  aggressive/conservative  problem  noticed  by  See  and  others  in  1997. 

Experiment  Manipulations 

As  mentioned  earlier,  arousal  is  one  potential  means  for  improving  vigilance 
performance.  Having  established^  vigilance  experiment,  four  manipulations  were  added  to 
explore  the  effects  of  arousal  on  vigilance  performance.  For  this  study,  the  subjects  were  evenly 
divided  into  two  groups  based  upon  age.  Group  1  was  comprised  of  subjects  age  35  and  older 
with  an  average  of 42.65  years,  while  Group  2  consisted  of  officers  age  34  and  younger  with  an 
average  age  of  27.8  years.  Within  each  group,  each  subject  was  randomly  assigned  to  one  of 
four  subgroups.  Subgoup  1  served  as  the  control  group  $nd  received  no  arousal  stimulus. 
Subgroup  2  received  an  arousal  stimulus  of  an  audio  warning  pue,  while  subgroup  3  received  a 
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visual  warning  cue.  Subgroup  4  received  a  combination  visual  and  audio  cue.  The  following  2 
x  4  matrix  illustrates  the  division  of  groups  and  the  application  of  arousal  methods. 


No  Cue 

Audio 

Visual 

Visual  and  Audio 

Older  Group  (1) 

1-1 

1-2 

1-3 

1-4 

Younger  Group  (2) 

2-1 

2-2 

2-3 

2-4 

Figure  3-2  2x4  Experimental  Design  Matrix 


The  visual  arousal  cue  included  a  pop-up  window  that  warned  the  subject  of  potential 
information  warfare  attacks.  The  audio  arousal  cue  was  a  three  second  tone  emanating  from  the 
computer.  The  combination  signal  includes  both  elements  of  t|ie  visual  and  audio  arousal  cues. 
During  the  training  phase  of  the  experiment,  the  subjects  were  acquainted  with  the  cues  in  order 
to  ensure  there  was  no  misconception  of  what  each  meftnt.  Finally,  the  computers  were 
programmed  to  not  make  any  sounds  or  provide  any  kind  of  pop-up  windows,  which  could  be 
misconstrued  by  the  subjects  as  false  warnings. 


Pilot  Study 

Before  conducting  the  actual  experiment,  a  pilot  study  was  conducted  to  validate  the 
design  of  the  experiment  and  its  measures.  For  the  pilot  study,  10  company  and  field  grade  Air 
Force  officers  were  the  subjects.  Upon  analysis  of  the  data  provided  by  the  shortened  study, 
effective  measures  were  determined  for  describing  the  hypotheses  listed  in  Chapter  2. 
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Additionally,  the  pilot  subjects  addressed  ergonomic  considerations  and  minor  display  changes 
were  made  to  the  program.  Overall  the  pilot  study  data  suggested  the  experiment  design  and 
data  collection  measures  were  sound. 


Hypothesis  Measures 

The  primary  source  of  data  collection  resides  within  the  DDD  system.  Statistics  on 
subject  performance  such  as  errors  detected,  time  of  dejection,  incorrect  detections,  and 
confidence  levels  are  recorded  by  the  system.  Additionally,  a  questionnaire  was  used  to 
determine  overall  arousal  levels  within  each  subject.  These  two  sources  of  data  provide  the  basis 
of  the  study’s  hypothesis  measures.  The  following  table  represents  the  measures  for  each 


hypothesis. 


Hypothesis 

Measures) 

Hl{a,b,c):  The  stimuli  of  audio  and  visual  clue 
signals  will  have  a  positive  affect  on  arousal. 

Arousal  level  from  questionnaire  compared  to 
the  type  of  stimuli  presented 

H2:  Older  age  will  limit  the  arousal  effects  of 
computer  generated  cue  signals. 

Self  reported  arousal  levels  compared  between 
groups  and  subgroups  based  upon  age  and 
arousal  method 

H3a:  Increased  arousal  positively  affects  the 
level  of  vigilance. 

Correct  detections  per  time  period  after  arousal 
cues  applied 

H3b:  Rates  of  vigilance  decrement  will  be 
lower  with  aroused  subjects  than  with  non- 
arousecf  subjects. 

Vigilance  curves  based  on  correct  detections 
per  time  period  Compared  to  self  reported 
arousal  levels 

H4a:  Increased  states  of  arousal  —induced 
vigilance  positively  affect  successful  error 
detectioa 

The  number  ofcorrect  detections  compared  to 
vigilance  levels 

H4b:  Improved  vigilance  decrement  has  a 
positive  association  with  successful  error 
detection. 

The  number  of  correct  detections  compared  to 
vigilance  curves  over  time 

H5a:  Increased  states  of  arousal-induced 
vigilance  are  at  the  expense  of  decreased 
productivity. 

Incorrect  detections,  time  taken  to  detect,  and 
number  of  times  detected  compared  against 
vigilance  levels 

H5b:  Improved  vigilance  decrement  has  a 
positive  association  with  productivity. 

Productivity  levels  per  time  period  compared 
to  vigilance  curves 

Table  3-2:  Hypothesis  Measures 
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Each  of  the  above  measures  is  based  upon  the  actions  pf  the  subject.  Also  collected  by 
the  DDD  system  are  confidence  levels  associated  with  each  subject’s  actions.  While  not  directly 
related  to  the  hypothesis  of  this  paper,  the  confidence  levejs  can  be  used  as  measures  of  the 
subject’s  intentions.  Further  explanation  of  the  measures  is  included  in  Chapter  4. 

Data  Analysis 

Once  the  measures  were  developed  and  the  data  compiled,  statistical  analysis  was 
conducted  for  each  measure.  Linear  regression  was  the  primary  statistical  tool  used  in  the  data 
analysis.  The  regression  allowed  an  easy  determination  of  the  factors  that  had  the  greatest 
influence  on  next  construct  in  the  model.  Additionally,  t-test  tools  were  used  for  determining  the 
significance  ofdata  comparisons  between  groups  1  and  2  and  ANOVAs  were  used  to  find  the 
differences  between  models  (Biros  1998;  Vankatesh,  Morris,  and  Ackerman  19xx).  Finally,  the 
vigilance  curves  were  statistically  analyzed  through  regressiojn  methods  developed  by  Methot 
and  Huitema  in  1998  (106). 

Summary 

The  experiment  design,  manipulations,  data  collection  tools,  and  hypothesis  measures 
developed  in  this  chapter  have  been  created  to  test  the  effects  of  arousal  on  vigilance  and  error 
detection.  Every  attempt  was  made  to  provide  a  realistic  experiment  with  direct  applicability  to 
information  warfare,  particularly  data  manipulation,  defense  prpcedures.  Chapter  4  will  discuss 
the  outcome  of  this  design  and  determine  which  hypotheses  were  supported. 
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CHAPTER  4:  RESULTS  AND  ANALYSIS 


Introduction 

This  chapter  presents  a  statistical  analysis  of  the  data  collected  from  the  experiment 
designed  in  the  previous  chapter.  It  will  examine  the  effects  of  stimuli  type  and  age  on  arousal 
levels  and  the  subsequent  impact  on  vigilance  and  vigilance  Recrement.  Further  analysis  will 
explore  how  changes  in  vigilance  affect  correct  detection  and  productivity.  These  findings  will 
be  related  to  the  hypotheses  generated  in  chapter  two  and  discussion  of  hypothesis  validity  will 
be  included. 

Experiment  Data 

The  DDD  system  provided  data  on  every  action  the  subjects  took  throughout  the 
experiment  and  questionnaires  were  used  to  obtain  other  data  on  each  subject.  The  first  step  in 
analyzing  the  data  provided  by  these  two  sources  was  to  identify  and  codify  the  hypothesis 
measures  from  the  raw  data.  Key  measures  used  in  this  study  were  age  (A),  arousal  level  (AL), 
vigilance  (V),  correct  detection  (CD),  productivity  (P),  and  vigilance  decrement  (VD).  Age  and 
arousal  level  values  were  taken  directly  from  the  questions  answered  by  each  subject.  Vigilance, 
correct  detection,  productivity,  and  vigilance  decrement  were  determined  based  upon  standards 
established  in  prior  vigilance  experiments  in  which  detections  over  time  served  as  the  baseline 
for  development  of  each  measure  (Parasuraman  and  Davies  1984;  Bisantz  2000;  See  et  al  1997). 
Together,  these  seven  variables  represent  each  portion  of  the  model  developed  in  chapter  2  and 
serve  as  the  basis  for  the  data  analysis  in  the  following  sections.  A  full  descriptive  analysis  of 
each  variable  and  any  applicable  interaction  is  located  in  Appendix  E.  Having  identified  and 
separated  the  key  measures,  the  analysis  could  begin. 
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Analysis  Tools 

In  analyzing  the  data,  the  primary  tool  used  was  regression.  The  theoretical  model  used 
in  this  study  followed  a  steady  progression  of  events  in  which  the  previous  event  influenced  the 
subsequent  events.  Therefore,  a  regression  analysis  provided  an  easy  method  for  determining 
which  factors  actually  influenced  the  next  variable  and  to  what  degree  the  influence  was  exerted. 
However,  since  the  subjects  had  been  split  into  two  groups  based  upon  age  levels,  a  comparison 
between  the  groups  was  also  required.  Using  an  analysis  technique  reported  by  Cohen  (1983) 
and  used  by  Venkatesh,  Morris  and  Ackerman,  a  t-test  of  the  beta  values  from  the  regression 
model  was  conducted  to  see  if  there  were  any  significant  differences  between  groups  (2000). 
Additionally,  an  Analysis  of  Variance  was  used  to  see  if  there  )vere  any  differences  between  the 
models.  The  following  sections  explore  the  results  of  the  data  analysis  in  relation  to  each 
hypothesis. 

Analysis  Results 

Effects  of  Stimuli  Types  and  Age  on  Arousal.  In  analyzing  the  data,  the  first  two 
hypotheses  (Hla,b,c  and  H2)  were  examined  together.  A  regression  model  was  built  for  each 
group  (old  vs.  young)  with  all  of  the  factors  that  may  have  contributed  to  the  changes  in  arousal 
states.  The  mean  arousal  for  group  1  was  4.45,  while  group  2  had  a  mean  arousal  of  5.27  on  a 
scale  of  1  to  7.  It  had  been  expected  that  stimuli  (S)  would  have  a  positive  affect  on  arousal  but 
that  it  would  be  limited  by  the  age  of  the  subjects.  For  this  model,  stimuli  was  reduced  to  the 
individual  arousal  methods  employed  in  order  to  determine  which  of  the  arousal  methods  had  the 
greatest  impact.  Additionally,  the  interaction  of  age  and  stimuli  was  also  explored.  The 
resulting  regression  model  was  as  follows: 
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AL  =  A  +  S  (audio,  visual,  audio/visual)  +  SA 
For  group  1,  the  model  yielded  an  R2  of  .149  with  visual  and  audio-visual  combination 
having  a  moderately  significant  positive  affect  on  arousal  and  audio  stimuli,  age,  and  the  age- 
stimuli  interaction  having  no  significant  effect  on  arousal  levels.  Group  2,  however,  had  an  R  of 
.472,  with  visual,  audio-visual  combination,  age  and  the  stimuli-age  interaction  having  a 
moderate  to  significant  effect  on  arousal  level.  Between  the  two  groups,  an  ANOVA  revealed  a 
significant  difference  (.000)  between  group  1  and  group  2,  while  a  t-test  revealed  a  significant 
difference  between  the  individual  betas  for  age  and  stimuli-age  interaction.  Table  4-1 
summarizes  the  results  of  the  regression,  ANOVA,  and  t-test  analysis  of  betas. 


R2 

Group  1 

Beta 

P 

R2 

Group  2 

Beta 

P 

Sianificance  of 
Difference 

Model  Beta 

0.149 

0.472 

0.000 

Audio 

0.243 

0.255 

0.450 

0.189 

0.252 

Visual 

0.853 

0.078 

1.091 

0.003 

0.176 

AV  Comb 

1.043 

0.047 

0.991 

0.012 

0.390 

Age 

0.166 

0.616 

0.851 

0.078 

0.032 

Sti-Age  Int 

.982 

0.325 

2.288 

0.041 

0.008 

Table  4-1 :  Factors  Affecting  Arousal 


HI  a:  The  arousal  method  of  audio  clue  signals  will  have  a  positive  affect  on  arousal. 
Hlb:  The  arousal  method  of  visual  clue  signals  will  have  a  positive  affect  on  arousal. 
Hlc:  The  arousal  method  of  audio  and  visual  combination  clue  signals  will  have  a 
positive  affect  on  arousal. 


The  results  of  the  data  analysis  indicate  no  support  for  Hla  in  either  group  for  audio 
stimuli  influence  on  arousal.  In  other  words,  the  subjects  did  not  have  their  arousal  levels 
significantly  increased  by  the  audio  arousal  cues.  Hlb  and  Hjc  did  receive  significant  support 
from  both  groups.  For  group  1,  visual  (.078)  and  the  audio-visual  combination  (.047)  were 
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moderately  significant  in  positively  affecting  the  subject’s  arousal  level.  For  group  2  the  results 
were  the  same,  but  with  a  strong  significance  for  both  visual  (.003)  and  audio-visual  combination 
(.012).  The  regression  models  for  both  groups  indicate  that  the  subgroups,  which  received  visual 
and  audio-visual  combination  arousal  methods,  had  the  greatest  influence  on  arousal  levels.  The 
differences  between  the  group’s  betas  were  not  significant  for  any  of  die  stimuli  types. 

HI:  Older  ages  will  limit  the  arousal  effects  of  computer  generated  cue  signals 

For  H2,  however,  the  results  are  mixed.  Group  1  had  no  significant  relationship  between 
age  or  sti-age  interaction  and  arousal.  In  contrast,  for  group  2,  age  (.032)  and  sti-age  interaction 
(.008)  had  a  strong  significant  effect  on  arousal.  The  implications  for  group  2,  include  the 
possibility  that  while  age  had  a  negative  association  with  arousal,  the  stimuli  type  and  sti-age 
interaction  influence  on  arousal  were  great  enough  to  offset  any  negative  impacts  of  age.  The 
difference  between  the  group’s  betas,  determined  by  the  t-test,  was  significant  at  .032  for  age  and 
.008  for  the  sti-age  interaction,  thereby  indicating  that  the  effects  of  age  and  the  sti-age 
interaction  varied  between  groups.  Finally,  group  1  was  most  impacted  by  the  combination  of 
audio  and  visual  stimuli,  while  group  2  was  influenced  by  both  visual  and  the  combination 
stimuli,  as  indicated  by  the  betas  fpr  the  different  arousal  methods. 

Effects  of  Arousal  on  Vigilance.  Vigilance,  for  this  study,  is  the  ability  to  detect  errors  in 
the  data  over  time.  Figure  4-1  illustrates  the  average  vigilance  curve  for  each  subgroup  for  this 
experiment.  These  curves  indicate  the  subject’s  ability  to  detect  errors  over  time.  They  also 
demonstrate  vigilance  decrement.  Vigilance  decrement,  for  this  study,  is  the  rate  of  decrease  in 
vigilance.  For  figure  4-1,  a  more  sharply  declined  slope  indicates  a  higher  vigilance  decrement, 
while  a  lower  vigilance  decrement  is  indicated  by  a  shallow  or  slightly  decreasing  slope.  This 
figure  will  be  referred  to  for  the  following  data  analysis  sections. 
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Figure  4-1:  Vigilance  Curves  for  Groups  1  and  2 


It  was  expected  that  once  aroused,  a  subject  would  ppssess  a  higher  state  of  vigilance 
than  if  not  aroused.  Therefore,  the  second  regression  analysis  centered  on  finding  the  factors  that 
influenced  the  level  of  vigilance  within  a  subject.  For  this  and  future  models  the  arousal 
methods  are  grouped  together  as  a  single  stimuli  factor.  The  regression  equation  included: 

V  =  S  +  A  +  AL  +  S  A 

An  interaction  component  between  stimuli  and  age  (SA)  was  also  initially  considered. 
The  above  equation  provided  an  R2  of .  1 19  for  group  1  and  a  somewhat  higher  value  of  .271  for 
group  2.  Additionally,  an  ANOVA  established  a  significant  difference  between  the  models  of 
.000  and  the  t-test  determined  a  significant  difference  between  the  beta  for  age  (.051)  and  sti-age 
interaction  (.012).  Table  4-2  summarizes  the  results  of  the  regression  analysis,  ANOVA,  and  t- 
test  for  H3a. 
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R2 

Grouo  1 

Rfita 

P 

R2 

Group  2 

Beta 

_ e_ 

Sianificance  of 
Difference 

Model  Beta 

0.119 

0.271 

0.000 

Stimuli 

-0.256 

0.774 

0,761 

0.068 

0.406 

Age 

0.069 

0.829 

-0.819 

0.156 

0.051 

-SiUAgeJnt 

(L073 

0.940 

1.720 

0.198 

0.012 

Arousal 

0.005 

0.964 

-0.029 

0.793 

0.113 

Table  4-2  Factors  Affecting  Vigilance 


H3a:  Increased  arousal  has  a  positive  association  with  the  level  of  vigilance. 

For  H3a,  there  was  no  indication  that  arousal  was  associated  with  vigilance.  However,  it 
should  be  noted  that  while  arousal  itself  did  not  have  significant  impact,  stimulus  type  did  have  a 
moderate  positive  association  with  vigilance  at  a  significance  of  .068  for  group  2.  Similar  to  H2, 
the  high  positive  beta  for  the  stimuli  factor  suggests  that  at  least  for  group  2,  the  subjects  who 
received  the  visual  and  audio-visual  combination  arousal  methods  did  experience  a  positive 
influence  on  their  vigilance  levels.  While  this  moderate  influence  occurred  for  group  2,  the 
difference  between  group  2  and  group  1  was  insignificant,  thereby  demonstrating  how  limited 
the  influence  actually  was. 

Again,  in  Figure  4-1  an  ANOVA  of  the  curves  indicated  a  significant  difference  between 
the  subjects  receiving  the  audio-visual  stimuli  and  those  who  received  no  stimuli  for  both  group 
1  and  group  2.  Additionally,  group  2  also  showed  a  significant  difference  between  subjects 
receiving  visual  stimuli  and  those  who  received  no  stimuli.  These  results  will  be  discussed 
further  in  the  following  section. 


Effects  of  Arousal  on  Vigilance  Decrement.  The  next  analysis  conducted  was  on  how 


arousal  influenced  vigilance  decrement.  It  was  expected  that  arousal  would  cause  a  continuously 


higher  state  of  vigilance  thereby  reducing  the  vigilance  decrement  phenomenon.  However,  H3a 
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indicated  that  arousal  had  not  affected  vigilance.  Therefore,  the  regression  model  included 
arousal,  stimuli,  age,  and  the  sti-age  interaction  in  order  to  determine  other  possible  influences 
on  vigilance  decrement.  The  final  model  was  as  follows: 

VD  =  S  + A+SA  +  AL 

This  model  provided  an  R2  of .  183  for  group  1  and  an  R2  of  .142  for  group  2.  The 
ANOVA  showed  only  a  modest  significant  difference  (.089)  between  the  models,  while  the  t-test 
of  beta’s  proved  significant  for  stimuli  (.032),  age  (.004),  and  the  sti-age  interaction  (.001). 

Table  4-5  shows  the  results  of  the  regression  analysis. 


Significance  of 

Group  1 

- 

Group  2 

Qifference 

R2 

Beta 

P 

R2 

Beta 

P 

Model 

Beta 

0.163 

0.442 

0.089 

Stimuli 

0.775 

0.367 

1.431 

0.182 

0.032 

Age 

0.152 

0.624 

-1.188 

0.058 

0.004 

Sti-Afe4nt 

-&408 

0.662 

2.526 

0.082 

0.001 

Arousal 

0.192 

0.040 

-0.131 

0.280 

0.337 

Table  4-3:  Factor?  affecting  Vigilance  Decrement 


H3b:  Rates  of  vigilance  decrement  will  be  lower  with  groused  subjects  than  with  non - 
aroused  subjects. 

H3b  suggested  that  the  rate  of  vigilance  decrement  would  be  lower  with  aroused  subjects 
than  with  non-aroused  subjects.  Group  1  found  that  arousal  had  a  strong  influence  (.040)  while 
group  2  experienced  no  significant  positive  affect  on  vigilance  decrement  by  arousal.  Instead, 
vigilance  decrement  for  group  2  was  moderately  impacted  by  age  (.058)  and  the  sti-age 
interaction  (.082).  These  results  indicate  that  for  the  oldey  group  arousal  did  influence  the 
vigilance  decrement  while  for  the  younger  group  other  factqrs  such  as  age  and  its  interaction 
with  stimuli  had  more  of  an  affect.  As  a  result,  only  group  1  provides  support  for  H3b. 
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In  figure  4-1,  the  vigilance  decrement  can  be  seen  as  the  declining  slope  of  the  vigilance 
curve  over  time.  For  group  1  and  group  2,  the  subjects  who  received  the  audio-visual 
combination  stimuli  had  the  lowest  rates  of  vigilance  decrement  as  indicated  by  the  shallow 
sloping  curve.  An  ANOVA  of  the  different  curves  for  group  1  showed  a  significant  difference 
(.01 1)  between  1-4  (audio-visual)  and  the  control  group  1-1  (no  stimuli).  All  other  subgroups  for 
group  1  were  deemed  to  have  no  significant  difference  compared  to  the  control  group. 
Additionally,  for  group  2,  the  ANOVA  found  subgroup  2-3  (visual)  and  2-4  (audio-visual)  to  be 
significantly  different  than  the  control  group  (no  stimuli)  at  a  level  of  .002  and  .005  respectively. 

Effects  of  Arousal  induced  Vigilance  and  Vigilance  Decrement  on  Correct  Detects.  The 
relationship  between  arousal,  vigilance,  vigilance  decrement  and  correct  detection  was  expected 
to  be  positive.  In  order  to  determine  how  the  relationship  existed  the  following  regression  model 
was  developed: 

CD  =  S  +  A  +  S  A  +  AL  +  V  +  AV  +  VD 

This  model  contains  the  factors  of  stimuli,  age,  sti-age  interaction,  arousal,  and  vigilance, 
arousal-vigilance  interaction,  and  vigilance  decrement.  The  stimulus,  age,  and  sti-age  interaction 
factors  were  included  due  to  the  weak  association  of  arousal  to  yigilance  demonstrated  above.  It 
seemed  possible  that  in  addition  to  arousal,  other  factors  such  as  an  arousal-vigilance  (AV) 

2 

interaction  could  have  also  influenced  the  number  of  correct  detects.  The  model  produced  an  R 
of .  144  for  group  1  and  a .  148  for  group  2.  An  ANOVA  showed  a  modest  difference  between 
the  group  1  and  group  2  of  .073  and  the  t-test  only  showed  a  significant  difference  between  the 
factors  age  (.074),  sti-age  interaction  (.003),  arousal  (.056),  and  vigilance  (.078). 
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Sianificance  of 

Group  1 

Group  2 

Difference 

R2 

Beta 

P 

R2 

Beta 

P 

Model  Beta 

0.144 

0.148 

0.073 

Stimuli 

0.231 

0.800 

0.227 

0.088 

0.110 

Age 

-0.239 

0.463 

0.075 

0.449 

0.074 

Sti-Agefnt 

0.306 

0.756 

-0.342 

0.211 

0.003 

Arousal 

-0.550 

0.040 

0.126 

0.585 

0.056 

Vigilance 

1,155 

0.044 

0.354 

0.525 

0.078 

Vigilance  Dec 

0.276 

0.208 

0.359 

0.064 

0.428 

Arous-Vig  Int 

1.125 

0.013 

1.136 

0.076 

0.387 

Table  4-4:  Factors  Affecting  Correct  Detection 


H4a:  Increased  states  of  arousal  induced  vigilance  have  a  positive  association  with 
sttecessfid  error-detection. 

H4b:  Improved  vigilance  decrement  has  a  positive  association  with  successful  error 
detection. 

For  H4a,  significant  support  was  found  to  indicate  that  vigilance  did  have  a  positive 
association  with  error  detection  for  group  1.  Vigilance  (.014),  arousal  (.040),  and  the  arousal- 
vigilance  interaction  (.013)  all  indicate  a  moderate  to  strong  significance  for  influencing  error 
detection  for  group  1.  In  contrast,  group  2  only  recorded  a  moderate  significance  (.076)  for  the 

i 

arousal-vigilance  interaction  influence  on  error  detection.  The  ANOVA  registered  a  moderate 
difference  (.073)  between  the  models  and  the  t-test  indicated  no  significant  difference  between 
the  any  of  the  factors  for  each  group.  These  results  suggest  that  for  group  1,  higher  levels  of 
vigilance,  as  indicated  by  higher  Beta  values,  do  positively  affect  the  error  detection  ability  of 
the  subjects.  Therefore,  H4a  can  only  be  partially  supported  by  the  findings  for  group  1. 

Referring  back  to  figure  4-1,  the  vigilance  curves  essentially  show  the  correct  detection 
of  errors  over  time.  As  noted  above,  the  vigilance  levels,  especially  those  of  the  subjects  who 
received  the  audio-visual  combination  stimuli  (1-4  and  2-4),  experienced  significantly  less 
vigilance  decrement,  thereby  allowing  better  error  detection  over  time.  In  this  study,  subgroups 
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1-4, 2-3,  and  2-4  represent  visual  and  audio-visual  combination  stimuli.  Their  graph  and  t-test 
analysis  indicate  that  they  had  the  best  influence  on  error  detection. 

For  H4b,  significant  support  was  found  to  suggest  that  improved  rates  of  vigilance 
decrement  would  influence  error  detection  for  group  2.  The  regression  model  seems  to  indicate 
that  at  least  at  a  moderate  level  of  significance  (.068),  error  detection  can  be  improved  by  a 
reduction  in  the  rate  of  vigilance  decrement.  Group  1,  however,  did  not  see  any  significant 
influence  on  error  detection  from  vigilance  decrement.  Therefore,  H4b  is  only  partially 
supported. 

Effects  of  Arousal  induced  Vigilance  and  Vigilance  Decrement  on  Productivity.  While 
higher  states  of  vigilance  were  expected  to  improve  error  detection,  the  opposite  was  expected  to 
be  true  for  productivity.  Improved  vigilance  decrement  was  also  anticipated  to  aid  in  error 
detection,  but  without  causing  the  undesired  effect  of  reducing  productivity.  The  effects  of 
vigilance  and  vigilance  decrement  on  productivity  are  reflected  in  the  following  model: 

P  =  S  +  A  +  SA  +  AL  +  V  +  VD  +  AV 

Similar  to  H4(a,b)  the  factors  of  stimuli,  age,  sti-^ge  interaction,  arousal,  arousal- 
vigilance  interaction  were  included  to  help  explain  productivity.  For  group  1,  the  model 
produced  an  R2  of  .094,  while  for  group  2  it  gave  and  R2  of  .085.  The  ANOVA  showed  a 
moderate  difference  between  models  (.063)  and  the  t-test  only  showed  a  significant  difference 
between  the  sti-age  interaction  and  vigilance  factors.  Table  44  shows  the  results  of  the 
regression  and  t-test  analysis. 
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Significance  of 

Group  1 

Group  2 

Difference 

Rz 

-Beta 

R2 

-Beta 

_ e_ 

Model  Beta 

0.094 

0.085 

0.063 

-Stimuli 

Q.219 

9915 

0.077 

0.572 

0.687 

Age 

0.209 

0.533 

9.063 

0.537 

0.396 

-Sti=Agelnt 

-0  353 

9J27 

0.188 

0.761 

0.056 

Arousal 

0.449 

0.102 

0.271 

0.259 

0.433 

VigilanceJDec 

-QJL53 

9.495 

0903 

0.989 

0.375 

Vigilance 

-0.340 

0.475 

1.081 

0.063 

0.079 

AtousA/ig  Int 

=0.923 

9947 

-1.353 

0.042 

0.353 

Table  4-4:  Factors  Affecting  Productivity 


H5a:  Increased  states  of  arousal-induced  vigilance  have  a  negative  association  with 
- productivity . 

HSb:  Vigilance  decrement  has  a  positive  association  with  productivity. 

The  regression  model  indicates  that  for  group  1,  vigilance  had  significant  (.020)  negative 
influence  upon  productivity.  In  contrast,  however,  for  group  2,  vigilance  had  a  moderately 
significant  (.074)  positive  influence  upon  productivity.  The  arousal-vigilance  interaction  was 
found  to  be  significant  for  both  group  1  and  group  2  at  a  significance  level  of  .047  and  .042 
repesctively.  These  results  suggest  that  for  the  older  group,  the  more  vigilant  a  subject  became 
the  more  likely  they  were  to  develop  unproductive  behaviqrs  such  as  incorrectly  identifying 
errors  that  did  not  exist  or  simply  spending  too  much  time  analyzing  the  errors  that  did  exist. 
Interestingly,  group  2  actually  increased  in  productivity  with  their  increased  vigilance  as 
indicated  by  the  positive  beta  value.  Therefore,  for  H5a  the  data  analysis  only  supports  the 
hypotheses  for  group  1.  H5b  was  not  supported  by  either  group  as  vigilance  decrement  had  no 
significant  influence  on  productivity. 
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Application  to  Model 

The  regression  analysis  performed  above  was  based  directly  on  the  theoretical  model 
generated  in  chapter  2.  The  following  illustration.  Figure  4-1,  summarizes  the  influence  each 
construct  had  on  subsequent  constructs. 


Figure  4-2:  Weighted  Theoretical  model  of  Arousal-Vigilance  Interaction 


The  illustration  demonstrates  how  significant  the  support  was  for  each  hypothesis  based 
upon  the  different  groups.  With  the  exception  of  H3a  and  H5b,  each  hypothesis  received 
moderate  to  strong  support  from  at  least  one  of  the  two  groups. 

Summary 

The  data  analysis  conducted  above  indicated  the  influence  each  of  the  constructs  had 
upon  one  another.  Additionally,  the  results  tended  to  support  either  in  part  or  completely  the 
hypotheses  generated  in  chapter  2,  with  the  exception  of  hypothesis  3.  A  discussion  of  these 
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results  as  well  as  complete  review  of  the  implications,  applications,  and  limitations  of  this  study 
will  be  discussed  in  chapter  5. 
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CHAPTERS:  CONCLUSIONS  AND  RECOMMENDATIONS 


Introduction 

The  previous  four  chapters  outlined  the  research  question,  explored  previous  research, 
developed  new  hypotheses  and  a  theoretical  model,  designed  an  experiment,  and  collected  and 


analyzed  the  data.  This  chapter  will  discuss  the  results  of  this  study  and  compare  them  to 
previous  work  accomplished  in  the  vigilance  field.  Additionally,  the  limitations  of  this  study 


will  be  addressed.  Finally,  the  application  to  the  Air  Force  and  areas  of  future  study  will  be 


offered. 


Observations 

Hypothesis  l(a.b.cl  The  first  hypothesis  stated  an  expectation  of  a  positive  relationship 
between  stimuli  and  arousal.  For  this  study,  Hlb  and  Hlc  were  both  strongly  supported  by  both 
groups  at  a  moderate  to  high  degree  of  significance  (G1  =  .078  and  .047,  G2  =  .003  and  .012). 
These  findings  are  consistent  with  previous  work  accomplished  on  arousal  (Blocher  et  al  1986; 
Tipples  and  Sharma  2000).  What  made  this  study  unique  was  that  it  considered  age  as  well  as 
stimuli  type.  The  regression  analysis  conducted  in  chapter  4  strongly  indicated  that  the  type  of 
stimuli  did  matter  for  both  the  older  group  as  well  as  for  the  younger  group.  The  regression  also 
indicated  that  for  the  older  group,  audio  and  visual  combination  stimuli  was  more  effective  than 
any  of  the  signals  individually,  while  for  the  younger  group  the  visual  stimuli  alone  was  the  most 
effective.  This  type  of  result  is  not  precedented  in  previous  research  as  few  studies  used  a  varied 
type  of  arousal  mechanism  (Mehrabian  1995).  Nevertheless,  hypotheses  lb  and  lc  were 
supported  by  the  data. 
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The  significance  of  these  findings  revolves  around  the  distinction  of  method  type  when 
arousing  a  subject.  Older  subjects,  while  less  significantly  aroused  then  younger  subjects 
overall,  seemed  to  respond  best  to  a  combination  of  audio  and  visual  signals.  In  contrast,  the 
younger  subjects  responded  better  to  visual  signals.  Both  groups,  however,  did  not  respond  to 
simple  audio  stimuli.  These  results  indicate  that  system  developers  who  are  interested  in 
arousing  the  system  operators  in  cases  of  information  warfare  attacks  must  closely  understand 
who  the  operators  are  and  then  apply  the  most  effective  means  available.  In  short,  effectiveness 
of  arousal  methods  may  depend  upon  the  intended  audience.  In  this  case,  age  made  a  difference 
in  arousability. 

Hypothesis  2.  The  second  hypothesis  was  closely  related  to  the  first  and  was  primarily 
designed  to  see  if  age  could  impact  or  in  some  way  limit  arousal.  Group  1  had  no  significant 
impact  for  age  upon  arousal.  Group  2,  however,  found  that  age  did  have  a  significant  negative 
influence  (.078)  on  arousal.  This  result  is  in  disagreement  with  the  previous  work  of  Dittmar  and 
Warm  (1993)  and  Pigeau  and  Angus  (1995)  who  all  found  that  the  more  age  a  subject  had,  the 
less  they  were  able  to  be  aroused.  For  this  study,  the  data  suggests  that  the  younger  group, 
whose  age  had  a  negative  impact  on  arousal,  may  have  been  limited  in  their  arousability  by  their 
age.  Regardless,  partial  support  for  H2  was  found  in  the  data  from  group  2. 

This  finding  suggests  that  the  previous  studies  may  not  have  considered  the  arousal 
method  when  concluding  that  age  limits  arousal.  In  light  of  the  findings  in  Hlb  and  H2c,  it  may 
stand  to  reason  that  stimuli  type  is  more  important  in  determining  whether  or  not  a  subject  will 
be  aroused  than  the  age  of  the  subject.  The  regression  model  in  Table  4-1  supports  this  concept 
in  the  sense  that  stimuli  type  was  more  significant  in  explaining  arousal  than  age. 
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Hypothesis  3(a.b).  For  this  study,  arousal  did  not  have  any  significant  impact  on 
vigilance  for  either  group.  This  finding  was  unexpected  as  it  contradicted  many  previous  studies 
(Parasuraman  and  Davies  1984;  Singh  and  Molloy  1993;  Paus  and  Zatorre  1997).  In  each  of 
these  studies,  some  form  of  stimuli  increased  the  arousal  level  within  the  subjects  to  a  point  were 
vigilance  had  been  positively  affected.  In  this  study,  arousal  itself  had  no  significant  impact  on 
vigilance,  yet  the  stimuli  itself  did  for  group  2.  In  particular,  the  visual  and  audio-visual 
combination  seemed  to  be  the  most  effective  in  relation  ship  to  vigilance.  Therefore,  while  no 
support  for  H3a  was  detected,  an  encouraging  discovery  of  the  impact  of  stimuli  type  on 
vigilance  was  important. 

Of  greater  value  to  vigilance  research  may  have  been  the  affect  of  arousal  on  vigilance 
decrement.  While  previous  studies  indicated  arousal  would  affect  vigilance,  none  discussed 
arousal’s  direct  impact  on  vigilance  decrement.  Parasuraman  and  Davies  suggested  that  arousal 
would  impact  vigilance  levels  but  do  little  for  vigilance  decrement  (1984).  In  this  study,  H3b 
posited  that  arousal  would  have  a  positive  affect  on  decreasing  the  rate  vigilance  decrement, 
particularly  over  an  extended  period  of  reapplication.  For  group  1  this  effect  is  suggested  by  the 
results  of  the  regression  analysis.  Arousal  significantly  influenced  (.040)  the  improvement  in 
vigilance  decrement.  This  result  indicates  that  the  vigilance  decrement  may  be  able  to  be 
averted,  at  least  for  short  periods,  if  the  right  kind  of  arousal  method  is  continuously  reapplied  to 
the  subject. 

Hypothesis  4(a.b).  Consistent  with  previous  work  dating  back  to  the  first  vigilance 
studies  of  N.  H.  Mackworth  in  1950,  this  study  found  that  vigilance  had  a  significant  (.014) 
affect  on  error  detection,  if  only  for  the  older  group  (Parasuraman  and  Davies  1984,  Dittmar  and 
Warm  1994).  This  result,  although  expected,  was  not  as  dramatic  as  hoped.  Group  2,  which  had 
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the  highest  average  detection  rate,  was  not  significantly  influenced  by  vigilance.  More 
important,  however,  may  be  the  effect  of  the  arousal-vigilance  interaction  upon  error  detection. 
Both  groups  experienced  a  positive  influence  (group  1:  .013  and  group  2:  .076)  upon  error 
detection  from  the  arousal-vigilance  interaction  factor.  This  finding  may  mean  that  artificially 
high  states  of  vigilance,  induced  by  arousal,  will  lead  to  improved  error  detection.  Further 
research  would  be  required  to  explore  that  possibility  as  the  results  for  H3a  indicated  arousal  had 
not  impacted  vigilance  to  any  significant  degree.  Nevertheless,  the  data  does  suggest  that 
support  exists  for  H4a. 

A  second  interesting  facet  of  error  detection  was  the  impact  of  vigilance  decrement. 
Hypothesis  H4b  suggested  that  improved  rates  of  vigilance  decrement  might  be  able  to  directly 
affect  error  detection.  For  group  2,  a  modestly  significant  (.06^)  influence  was  exerted  on  error 
detection.  Group  1,  however,  did  not  share  this  result.  These  findings  signify  that  vigilance 
alone  is  not  the  only  determinant  of  error  detection.  As  a  result,  partial  support  for  H4b  is 
suggested  by  the  regression  analysis. 

Hypothesis  5(a.bY  It  was  speculated  that  productivity  would  be  the  casualty  of  increased 
arousal  and  vigilance  (Biros  1998;  Hackley  and  Valle-Inclan  1999;  Mosier  et  al  1999).  For 
group  2,  higher  vigilance  did  have  a  moderately  significant  (.063)  negative  impact  on 
productivity.  This  study  defined  productivity  as  a  combination  of  incorrect  detects  and  time 
wasted  on  already  detected  errors.  Group  1  experienced  increases  in  the  unproductive  behaviors 
as  their  vigilance  increased.  Group  2,  however  did  not  have  the  same  problem.  In  actuality, 
their  higher  states  of  vigilance  actually  improved,  or  at  least  maintained  their  level  of 
productivity.  Consequently,  hypothesis  5a  was  only  partially  supported.  Like  H4a,  when 
considering  the  interaction  of  arousal  and  vigilance,  the  effect  on  productivity  changes.  Groupl 
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and  group  2  both  experienced  a  significant  negative  impact  uppn  productivity  from  the  arousal- 
vigilance  interaction  factor  (.047  and  .042  respectively).  This  result  indicates  that  a  decrease  in 
productivity  was  tied  to  arousal  induced  vigilance,  if  only  indirectly.  The  significance  of  this 
finding  reflects  the  discoveries  of  past  research  in  that  efforts  to  produce  higher  vigilance  and 
improved  error  detection  are  often  accompanied  by  a  decrease  in  productivity  for  the  subject. 

In  addition  to  arousal  and  vigilance,  vigilance  decrement  also  may  have  impacted 
productivity.  H4b  proposed  that  improved  vigilance  decrement  rates  could  positively  affect 
productivity.  For  this  study,  no  supporting  evidence  was  available. 

Summary.  The  subjects  in  groups  1  and  2  both  appear  to  be  influenced  by  arousal 
stimuli,  although  the  type  of  stimuli  most  effective  varied  between  groups.  Also  evidence  exists 
to  suggest  arousal  is  not  as  influenced  by  age  as  it  by  arousal  method.  Further  evidence  failed  to 
support  the  possibility  of  arousal  directly  influencing  vigilance  ^nd  only  moderate  support  exists 
to  suggest  that  it  influences  vigilance  decrement.  Similarly,  vjgilance  and  vigilance  decrement 
only  moderately  influence  error  detection  and  productivity,  while  the  interaction  of  arousal  and 
vigilance  appears  to  have  significantly  impacted  both  prror  detection  and  productivity. 

Limitations 

Arousal  Measuring  Methods.  The  unexplained  lack  of  influence  arousal  had  on 
vigilance  may  reflect  a  problem  in  the  reporting  of  arousal  by  the  subjects.  Although  the  survey 
technique  used  has  been  proven  in  previous  studies,  the  most  effective  way  to  measure  arousal  is 
through  medical  testing  equipment  such  as  the  EKG  or  heart  monitor  (Hackley  and  Valle-Inclan 
1999;  Paus  and  Zatorre  1997).  Such  equipment  can  measure  the  physiological  changes  within 
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the  human  body  and  thus  eliminate  the  need  to  rely  upon  the  subject  to  “remember”  mid  then 
quantify  how  he  or  she  felt. 

Training.  A  second  limitation  of  this  study  included  the  training  time  available  for  each 
subject  to  become  familiar  with  the  DDD  system.  In  reality  an  operator  would  be  expected  to 
thoroughly  know  and  understand  the  system  he  or  she  is  operating.  For  this  experiment,  each 
subject  received  30  minutes  of  classroom  instruction  and  20  minutes  of  hands-on  time  with  the 
system.  Survey  results  reveal  that  the  subjects  claimed  to  have  qnderstood  their  tasks  and  how  to 
use  the  system,  but  that  is  far  from  expecting  them  to  be  proficient.  A  subject  who  thoroughly 
understands  the  system  in  use  may  be  more  likely  to  detect  any  potential  problems. 

T  flhnratnry  Setting  The  third  limitation  for  this  study  was  the  laboratory  setting. 
Although  every  possible  measure  was  taken  to  make  the  simulation  as  real  as  possible,  the  fact 
remains  that  the  experiment  took  place  within  controlled  setting^.  The  subjects  could  not  feel  the 
pressures  of  a  wartime  exercise  or  the  possible  consequences  of  incorrect  decisions.  The  use  of  a 
scoring  system  may  have  mitigated  these  problems,  but  a  complete  duplication  of  a  real  world 
scenario  was  not  within  |he  time  and  means  of  this  study. 

Physical  Capabilities.  The  final  limitation  of  this  study  may  have  been  the  lack  of  testing 
for  sensory  perception  within  the  subjects.  This  study  relied  upqn  the  subjects  being  able  to  hear 
and  see  the  arousal  signals.  Given  the  range  in  ages  of  the  subjects,  it  may  have  been  possible 
that  some  subjects  were  not  able  to  detect  the  arousal  cues  due  to  poor  eyesight  or  hearing. 

Implications  for  the  Air  Force 

This  study  was  conducted  with  sponsorship  from  the  Air  Force  Office  of  Scientific 
Research  and  was  designed  to  explore  methods  for  improving  defenses  against  information 
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warfare  attacks.  In  this  light,  the  implications  for  the  Air  Force  primarily  revolve  around 
information  warfare  tactics  and  defenses,  but  could  be  extended  to  any  arena  using  automated 
decision  aids. 

The  first  implication  is  for  system  designers.  While  the  study  did  not  show  that  arousal 
could  affect  vigilance,  it  did  give  strong  indications  that  some  level  of  stimuli  did  have  an  affect 
on  vigilance.  System  designers  should  consider  incorporating  warning  signals  within  the  system 
that  can  be  selectively  initiated  in  the  event  of  an  information  warfare  attack.  Such  a  signal 
could  give  the  desired  “heads-up”  effect  that  can  help  operators  remain  vigilant  and  detect  data 
manipulations  before  they  can  do  any  harm. 

The  second  implication  is  for  information  warfare  defenses.  This  research  indicates  the 
possibility  for  command  and  control  system  operators  to  experience  vigilance  decrement  over 
time.  As  a  result,  information  warfare  operators  should  explore  methods  for  improving  the 
chance  that  errors  caused  by  data  manipulation  would  be  detected.  One  aspect  of  this  study 
looked  at  age  and  issues.  In  an  environment  where  information  jechnology  is  the  primary  means 
of  obtaining  decision-making  data,  the  younger  group  seemed  naturally  more  adept  at  detecting 
errors.  There  may  be  many  reasons  for  this  phenomenon  to  include  a  predisposition  for 
interpreting  computer  technology  due  to  a  greater  familiarity  with  computers  or,  as  this  study 
seems  to  indicate,  a  naturally  higher  state  of  vigilance.  Either  way,  the  results  of  this  study 
suggest  that  youthfolhess  may  help  in  error  detection.  However,  on  a  note  of  encouragement,  if 
older  personnel  are  required  to  operate  the  systems,  the  warning  stimuli  did  seem  to  improve 
their  ability  to  detect  errors. 
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Recommendations  for  Future  Research 

This  study  explored  a  very  limited  area  of  vigilance  and  arousal  research.  Many  other 
aspects  of  vigilance-arousal  interaction  remain  open  to  study.  First,  one  of  the  key  issues 
surrounding  arousal  is  the  length  of  time  arousal  is  effective.  The  point  at  which  the  problem  of 
habituation  occurs  is  not  well  addressed  in  the  literature  pn  arousal.  Similarly,  the  proper 
frequency  of  arousal  application  may  play  a  role  in  how  effective  arousal  is.  For  this  study  a 
stimuli,  if  applied,  was  applied  once  every  200  seconds.  Variations  of  that  may  produce  different 
results.  Finally,  it  may  be  interesting  to  explore  the  different  arousal  effects  on  gender.  Dittmar 
and  Warm  have  conducted  several  studies  on  vigilance  and  gender  and  have  suggested  a  very 
possible  link  between  arousal  cues  and  gender  (1993).  Research  into  gender  differences  could 
answer  many  questions  on  how  vigilance  can  be  affected  by  arousal  techniques.  Given  the 
findings  concerning  arousal-vigilance  interaction,  there  is  sigmfipant  reason  to  believe  that  future 
research  on  arousal  and  vigilance  may  yield  important  results. 

Other  areas  of  potential  study  include  the  interaction  of  arousal  and  confidence  on  error 
detection.  Evidence  from  this  study  suggests  an  inverse  relationship  exists.  While  it  was  not  a 
focus  of  this  experiment,  confidence  levels  of  the  subjects  were  collected  for  every  decision  they 
made.  Analysis  of  the  confidence  levels  seemed  to  indicate  that  when  arousal  and  vigilance  was 
high,  the  subject’s  confidence  in  their  decision  was  low.  A  study  could  be  developed  to  explore 
the  implications  of  such  a  relationship. 

Finally,  the  best  future  study  in  the  arousal-vigilance  research  would  include  a  multiple 
measure  of  arousal.  This  study  may  have  been  limited  on  the  ability  to  accurately  assess  a 
subject’s  arousal  level.  The  use  of  physiological  measures  as  well  as  self-reporting 
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questionnaires  may  prove  to  be  valuable  for  future  studies  of  arousal.  Once  accurately  measured, 
the  true  impact  of  arousal  may  be  found. 

Conclusion 

The  results  of  this  study  indicate  that  arousal  does  have  an  impact  on  vigilance.  In  an 
information  warfare  data  manipulation  environment,  the  ability  to  detect  errors  is  of  utmost 
importance.  Arousal-vigilance  research  may  provide  one  avenue  for  containing  the  threat  of 
data  manipulation. 


62 


Appendix  A:  Power  Analysis 


For  this  experiment  the  following  2-4  Matrix  describes  the  various  factors  and  levels: 


Older 

Age 

Visual 

Audio 

Visual/ 

Audio 

No  Cue 

Younger 
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Visual 

Audio 

Visual/ 

Audio 

No  C  ue 

Using  a  Power  Analysis  software  created  by  Dr.  Russ  Lenth  at  the  University  of  Iowa,  a 
power  analysis  was  performed  to  determine  the  proper  number  of  subjects  required  to  make  this 
study  statistically  valid.  The  following  sections  are  the  results  of  the  analysis. 

These  two  charts  depict  the  expected  power  of  a  sample  size  of  6  per  cell  (above  matrix) 
fora  total  number-of  48  subjects.  At  a  significance  level  of  .05,  foe  power  for  age  and  arousal 
method  is  .9977  and  .9993  respectively  (assuming  a  standard  deviation  of  1). 


Appendix  B:  Experiment  Scenario 


BACKGROUND: 

You  are  the  air  defense  commander  for  766th  Air  Defense  Unit  deployed  in  Northwest 
Taiwan.  The  766th  is  a  joint  air  defense  unit  that  integrates  tactical  ground  radar  units  and 
surface-to-air  missile  defense  units  into  a  single  weapon  system,  The  ADU  is  a  deployed  arm  of 
the  Air  Operation  Center  and  has  data  connectivity  with  the  AOC,  remote  radar  sites,  and  remote 
SAM  sites. 
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THE  PRESENT:  You  have  just  received  the  crew  changeover  briefing  where  you  received  the 
standard  mission  briefing.  Intelligence  briefing,  and  the  Rules  of  Engagement  briefing.  The 
following  is  a  summary  of  the  information  you  received: 

MISSION:  Defend  the  assigned  air  space  against  any  suspected  hostile  aircraft.  The  766th  is 
one  of  several  air  defense  units  dispersed  along  the  coast  of  Taiwan.  You  are  responsible  for  air 
surveillance,  track  identification,  and  weapon  interdiction.  The  commander  of  the  766th  is  also 
responsible  for  assigning  a  confidence  level  to  all  track  information  and  forwarding  that 
informajion  to  the  Air  Operation  Center. 

INTEL  BRIEF: 

In  early  July,  the  Chinese  government  declared  it  does  not  recognize  the  independence  of 
Taiwan,  as  declared  by  the  Taiwanese  government  this  past  June. 

In  response  to  this  declaration,  Taiwan  requested  and  received  military  support  from  the 
United  States.  This  support  consisted  of  two  naval  battle  groups,  the  regional  deployment  of  120 
fighter  and  support  aircraft,  and  the  local  deployment  of  5  new  Air  Defense  Units  with  remotely 
operated  radar  and  SAM  sites. 

The  deployment  was  completed  in  late  August.  Following  this  deployment,  China  threated 
that  if  allied  forces  were  not  withdrawn  by  the  first  of  September  then  China  would  reserve  the 
option  for  a  military  response.  Intel  sources  and  satellite  imagery  indicate  a  massive  Chinese  air 
assault  is  imminent. 

Intel  also  reports  that  the  Peoples  Republic  of  China’s  Information  Warfare  Force  (IWF) 

i 

have  been  probing  the  U.S.  forces  Wide  Area  Network.  The  IWF  technology  is  thought  to 
include  some  of  the  most  advanced  network  attack  and  information  manipulation  systems  in  the 
world.  The  Chinese  have  recently  demonstrated  a  successful  Information  Warfare  attack,  known 
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as  Strategic  Information  Manipulation  (SIM),  against  the  Taiwanese  government.  SIM  is  a 
technique  whereby  the  network  is  covertly  accessed  and  real-time  tactical  or  strategic 
information  is  manipulated  in  order  to  confuse  or  spoof  the  enemy 
RULES  OF  ENGAGEMENT:  By  the  order  of  the  President  of  the  United  States,  all  US 
military  forces  are  authorized  to  use  deadly  force  to  interact  hostile  aircraft  from  entering 
Taiwanese  airspace. 


66 


Appendix  C:  Training  Slides 
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attack  using  a  Surface  to  Air  Missile 
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If  an  enemy  enters  the  protected  air  space, 
you  will  lose  1  point  for  each  second  it 
remains  in  the  air  space.  « _ 
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Hands  On  Demonstration 


Practice  Simulation 
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Appendix  D:  Questionnaires 


Informed  Consent  Form 
Study  Overview 

Welcome  to  the  experiment.  The  following  is  a  general  description  of  the  study  and  a 
reminder  of  yeur  rights  as  a  potential  subject.  As  in  any  study,  your  participation  is  completely 
voluntary.  If  now,  or  at  any  point  during  the  study,  you  decide  that  you  do  not  want  to  continue 
^participating,  please  let  theexperimenter  know  and  you  will  be  dismissed  without  penalty.  Also, 
please  remember  that  your  name  will  not  be  associated  with  any  of  the  information  that  you 
provide  during  die  study.  All^ffte  information  you  provide  is  absolutely  anonymous  and 
confidential. 

In  this  study,  you  will  be  working  as  part  of  a  group  to  complete  a  mission  objective.  You 
-will  also  be  asked-to  complete  two  questionnaires  during  the  study.  You  will  first  be  given  a 
questionnaire  to  complete,  then  following  the  training,  you  will  be  given  the  second  second 
questionnaireto  complete.  The  experimenter  will  give  you  more  specific  instructions  later  in  the 
study.  If  you  have  any  questions  or  concerns  at  this  time,  please  inform  the  experimenter. 

For  further  information 

The  Air  Force  Institute  of  Technology  faculty  members  Responsible  for  conducting  this 
research  are  Maj .  Bavid  fiiros.  He  would  be  happy  to  addressatiy  of  your  questions  or  concerns 
regarding  this  study.  Maj.  Biros  can  be  reached  at  255-3636  ext  4578. 

If  you  would  like  to  participate  in  this  study,  please  sign  in  the  space  provided.  Your 
signature  indicates  that  you  are  aware  of  each  of  the  following;:  1)  the  general  procedure  to  be 
used  in  this  study,  2)  your  right  to  discontinue  participation  at  any  time,  and  3)  you  and  your 
name  will  not  be  associated  with  any  of  the  information  you  provide. 


Printed  Ni 


qpie: 


Signature: 


Date: 
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Survey  1 

Participant  Information  Sheet  Participant  # - 

INSTRUCTIONS 

This  is  a  short  two-part  survey  to  determine  the  demographic  information  of  the 
participants  in  this  research  as^weH  as  their  experience  level  with  computer  systems.  The  data 
collected  will  be  used  to  aid  in  the  evaluation  of  the  results  of  the  simulation.  All  information 
provided  will  be  kept  confidential  and  will  not  be  able  to  be  traced  back  to  the  participant. 

SECTION  1  -  Demographic  Information 

1.  Age _ 

2.  Rank _  Service  (USAF,  Army,  Navy) _ 

3.  AFSC _ _ _ _ _ 

4.  Number  of  years  served  in  current  AFSC _ 

5.  Total  number  of  years  served  in  the  military _ 

6.  Highest  Level  of  Education  (circle  one):  High  School,  Undergraduate,  Graduate,  Doctoral 

7.  Operational  experience  in  Combat/Hostile  Duty  Location  (yes/no) _ 

SECTION  II  -  Computer  Experience  an  Attitudes  (Circle  One) 

1.  Are  you  currently,  or  have  you  ever,  worked  in  a  computer  communications  position? 

Y  N 

2.  Do  you  consider  yourself  to  be  knowledgeable  about  computers?  Y  N 

3.  Are  you  familiar  with  how  a  computer  network  operates?  Y  N 

4.  Are  you  familiar  with  any  programming  languages?  Y  N 

5.  Which  computer  programs  do  you  use  on  a  frequent  basis _ 
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6.  Do  you  like  to  use  computers  to  conduct  work?  Y  N 

7.  Doyoufeel  comfortable  with  the  role  computers  play  in  today’s  Air  Force?  Y  N 

Why  or  why  not? 


SECTION  HI:  Computer  Beliefs 

Please  answer  all  of  the  questions  below.  Use  the  scale  provided  and  enter  the  number  that  best 
matches  your  beliefs. 

1  =  Strongly  Disagree;  2  =  Disagree;  3  =  Somewhat  Disagree;  4  =  No  opinion 
5  =  Somewhat  Agree;  6  =  Agree;  7  =  Strongly  Agree 

1 .  _ If  you  initiate  a  task  for  the  average  computer  system  to  perform,  the  computer 

system  will  finish  it  correctly. 

2.  _ I  believe  that  most  computer  systems  are  consistent. 

3.  _ Mqpt  computer  systems  are  reliable. 

4.  _ I  believe  that  most  computer  systems  are  technically  competent. 

5.  _ I  feel  I  can  depend  on  most  computer  systems. 

6.  _ I  can  trust  most  computer  systems. 
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Survey  2 


Participant  #: _ 

INSTRUCTIONS 

The  information  you  provide  will  be  kept  confidential.  In  addition,  your  identity  will  not  be 
linked  to  this  data.  The  information  collected  from  this  form  will  be  used  to  help  evaluate  the 
ADU  system  and  training  program. 

Definition:  Command  and  Control  (C2) 

Command  and  control  (C2)  describes  the  basic  job  of  the  militaty  battle  commander.  The  battle 
commander  is  responsible  for  directing  military  forces  to  accomplish  military  objectives  against 
an  adversary.  In  your  case,  this  is  air  space  defense  using  surface-to-air  missiles.  C2  objectives 
often  result  in  material  damage  and/or  human  casualties  to  both  the  adversary  and  friendly 
forces. 


Participant  Questionnaire 

Please  answer  allofthe  questions  below.  Use  the  scale  provided  and  enter  the  number  that  best 

matches  your  beliefs. 

X  =  Strongly  Disagree;  2  =  Disagree;  3  =  Somewhat  Disagree;  4  =  No  Opinion;  5  — 
Somewhat  Agree;  6  =  Agree;  7  =Strongly  Agree 


1 .  _  In  a  command  and  control  environment  like  described  in  the  scenario  brief,  I 

believe  computers  can  be  relied  upon  to  help  commanders  make  operational  decisions. 

2.  _  I  feel  I  can  depend  on  computer  systems  to  provide  timely  and  accurate 

information  to  battle  commanders  in  a  combat  situation. 

3.  _  In  a  command  and  control  setting  like  the  one  described  in  the  scenario,  I  feel  that 

I  can  adequately  trust  information  received  from  most  computer  systems. 

4.  _  I  believe  that  most  computer  systems  used  in  deployable  battle  cabs  are  secure 

enough  to  trust  in  combat  situations. 

5.  _  I  feel  most  computer  systems  used  in  command  and  control  units  are  dependable. 
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Survey  3  Participant  #: _ 

INSTRUCTIONS 

The  information  you  provide  will  be  kept  confidential.  In  addition,  your  identity  will  not  be 
linked  to  this  data.  The  information  collected  from  this  form  will  be  used  to  help  evaluate  the 
ADU  computer  system  and  training  program 


Please  answer  all  of  the  questions  below.  Use  the  scale  provided  and  enter  the  number  that  best 

matches  your  beliefs. 

1  =  Strongh  Disagree;  2  =  Disagree;  3  >*  Somewhat  Disagree;  4  =  No  Opinion;  5  =  Somewhat  Agree;  6  =  Agree;  7  “Strongly  Agree 


1. 

2. 

3. 

4. 

5. 

6. 
7. 


The  DDD  computer  system  is  predictable. 

The  DDD  computer  system  is  consistent. 

The  DDD  computer  system  is  technically  competent. 
The  DDjD  computer  system  has  integrity. 

The  EjDD  computer  system  is  reliable. 

The  DDD  computer  system  is  dependable 
I  can  trust  the  DDD  computer  system. 
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Training  Evaluation 

Please  cirde  the  correct  answer: 


1 .  The  role  of  the  Network  Security  Force  is  to _ 

a.  Monitor  the  network  only  • 

b.  Protect  the  network  only 

c.  Monitor  and  Protect  the  network 

d.  None  of  the  above 

2.  An  upside  “V”  shaped  icon  that  is  colored  red  represents  what  type  of  track? 

a.  Friendly 

b.  Hostile 

c.  Unknown 

d.  None  of  the  above 


3. 


Which  of  the  following  are  Iijformation  Warfare  tactics? 

a.  Denial  of  Service 

b.  Information  Manipulation 

c.  Hacking 

d.  All  of  the  above 


4.  The  main  components  qj" the  ADU  are _ 

a.  The  DDD  computer  system,  the  WAN,  and  the  sensor  sites 

b.  The  DDD  computer  system  and  the  Network  Security  Forces 

c.  The  Network  Security  Forces,  the  DDD  computer  system,  and  the  WAN 

d.  None  of  the  above 


5.  Track  identity  is  automatically  determined  by  DDD.  A  secondary  tpeans  by  which  you  can  verify  the  track 
identity  is  to _ 

a.  Send  a  request  to  the  AOC 

b.  Read  itjcoming  messages  from  the  sensor  sites. 

c.  None  of  the  above 

d.  All  of  the  above 
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Survey  4 

Post  Simulation  Evaluation  Sheet  Participant  # _ 

INSTRUCTIONS 

This  is  a  short  survey  to  assess  the  participant’s  reaction  to  the  simulation.  Please  circle  the 
correct  answer. 

1.  Were  the  instructions  clear  and  understandable?  Y  N 

2.  Was  the  simulation  easy  to  understand?  Y  N 

3.  Was  the  training  sufficient  for  you  to  play  the  game?  Y  N 

4.  Did  you  encounter  any  difficulties  in  following  the  instructions  for  the  game?  Y  N 

5.  Was  the  game's  operations  tempo  too  fast?  Y  N 

6.  Was  the  scenario  realistic?  Y  N 

7.  Arousal  is  defined  as  a  heightened  state  of  alertness  due  to  ap  external  stimuli.  On  a  scale  of 

1  to  7,  please  state  the  level  of  arousal  you  felt  when  alerted  to  a  network  attack. 

Lowest  Highest 

1  2  3  4  5  6  7 

8.  Did  the  warnings  cause  you  to  alter  your  strategies?  Y  N 

9.  Did  you  find  the  warnings  credible?  Y  N 

10.  Did  you  notice  any  errors  in  the  simulations?  Y  N 

If  so,  how  many?  _ 

Thank  you  for  participating  in  this  research.  Your  inputs  are  extremely  valuable. 
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Appendix  E  Statistical  Analysis  Tables 

Descriptive  Statistics 


Age- 


Groups 

Stimuli 

%  Cor 

Vig 

Arous- 

1  and  2 

Age  fnt 

Det 

Prod  Arous 

Dec 

Vig 

Vig  Int 

Mean 

34.33  158.483 

053 

0.33 

4.86 

0.33 

3.77 

19.22 

Std  Err 

0.62  4.04958 

0.03 

0.03 

0.09 

0.01 

0.16 

1.00 

Median 

430.00  168 

1.00 

0.00 

5.00 

0.33 

3.00 

15.00 

Mode 

27.00  120 

1.00 

0.00 

5.00 

0.25 

3.00 

15.00 

-Std-Dev 

-8.72  57.9812 

0.49 

0.47 

1.23 

0.18 

2.26 

14.03 

Sam  Var 

76.07  3361.82 

0.24 

0.22 

1.52 

0.03 

5.09  196.95 

-Kurtosis 

-0.64  -0.3773 

-1.74 

-1.51 

-0.41 

-0.57  -0.40 

0.61 

Skew 

0.73  -0.4421 

-0.52 

0.71 

-0.53 

0.18 

0.62 

1.06 

Age 


vfvtiu 

Stimuli 

%Cor 

Vig 

Arous- 

1-1 

Age 

Int 

Det 

Prod  Arous 

Dec 

Vig 

Vig  Int 

-Mean 

39.39 

40.5 

044 

0.39 

4.56 

0.14 

1.44 

6.94 

Std-Eir 

1.15  0.90116 

0.12 

0.12 

0.23 

0.02 

0.18 

0.95 

Median 

40.50 

41 

0.00 

0.00 

4.00 

0.17 

2.00 

8.00 

Mode 

42.00 

42 

0.00 

0.00 

4.00 

0.17 

200 

8.00 

Std  Dev 

459 

-3.8233 

0,51 

0.50 

0.98 

0.10 

0.78 

4.04 

SamVar 

23.90  14.6176 

0.26 

0.25 

0.97 

0.01 

0.61 

16.29 

-Kurtosis 

-3.14005816 

-200 

-1.99 

-0.92 

0.08  -0.45 

-0.65 

Skew 

-0.750.61099 

0.24  ( 

0.50 

0.24 

0.42 

-1.03 

-0.59 

Age- 

Group 

otmtun 

%Cor 

Vig 

Arous- 

1-2 

Age 

ini 

Det 

Prod  Arous 

Dec 

Vig 

Vig  Int 

Mean 

43.11  -86,2405 

047 

0.47 

3.95 

0.22 

2.47 

9.68 

Std-Err 

1:73 

34536 

0.12  , 

0.12 

0.30 

0.02 

0.31 

148 

Median 

4200 

84 

0.00 

0.00 

4.00 

0.25 

3.00 

9.00 

Mode 

36.00 

72 

0.00 

0.00 

3.00 

0.25 

3.00 

12.00 

-Std-Dev 

7.53 

450539 

0.51 

0.51 

1.31 

0.10 

1.35 

6.47 

SamVar  56.65 

226.62 

0.26 

0.26 

1.72 

0.01 

1.82 

41.89 

Kurtosis 

=029 

0.2867 

-204 

-2.24 

-1.14 

-0.43 

-1.38 

-0.24 

Skew 

0.91 

0.90827 

0.11 

0.11 

0.27 

-0.72 

-0.39 

047 

Age- 

Group 

Stimuli 

%Cor 

Vig 

Arous- 

1-3 

Age 

Int 

Det 

Prod  Arous 

Dec 

Vig 

Vig  Int 

Mean 

45.88  137.654 

0.46 

0.54 

4.35 

0.28 

3.12 

13.31 

Std-Err 

0.96  207907 

0.10 

0.10 

0.28 

0.02 

0.27 

1.38 

Median 

44.00 

132 

0.00 

1.00 

5.00 

0.25 

3.00 

15.00 

Mode 

51.00 

153 

0.00 

1.00 

5.00 

0.25 

3.00 

18.00 

Std  Dev 

4.89  14.6804 

0.51 

0.51 

1.44 

0.12 

1.37 

7.01 
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SamVar  23.95  215.515  0.26  0.26  2.08  0.01  1.87  49.18 

Kurtosis  -1.96  -1.9605  -2.14  -2.14  -0.78  -0.39  -0.87  -1.18 

Skew  0.00  0.00232  0.1^  -0.16  -0.75  -0.43  0.08  0.11 


Age- 


Group 

Stimuli 

%  Cor 

Vig 

Arous- 

1-4 

Age  Int 

Det 

Prod  Arous 

Dec 

Vig 

Vig  tnt 

Mean 

41.28  165.12 

0.50 

0.36 

4.88 

0.25 

2.56 

12.84 

Std  Err 

1.04  4.16666 

0.10 

0.10 

0.25 

0.02 

0.14 

1.08 

Median 

40.00  160 

1.00 

0.00 

5.00 

0.25 

3.00 

15.00 

Mode 

35.00  140 

1.00 

0.00 

6.00 

0.25 

3.00 

18.00 

Std  Dev 

5.21  20.8333 

0.50 

0.49 

1.27 

0.08 

0.71 

5.38 

Sam  Var 

27.13  434.027 

0.25 

0.24 

1.61 

0.01 

0.51 

28.97 

Kurtosis 

-1.64  -15392 

-1.98 

-1.76 

-1.44 

2.07 

5.83 

-0.60 

Skew 

-0.02  -0.0194 

-0.43 

0.62 

-0.56 

-1.42 

-2.11 

-0.65 

Age- 

VivUU 

Stimuii 

%  Cor 

Vig 

Arous- 

2-1 

Age  Int 

Det 

Prod  Arous 

Dec 

vig 

Vig  Int 

Mean 

25.67  428.333 

0.62 

0.38 

4.19 

0.39 

3.48 

14.10 

Stef  Err 

0.32  1.59364 

0.11 

0.11 

0.25 

0.05 

0.32 

1.46 

Median 

26.00  1-3Q 

1.00 

0.00 

4.00 

0.42 

4.00 

12.00 

Mode' 

24.00  120 

1.00 

0.00 

3.00 

0.42 

5.00 

12.00 

Std  Dev 

1.46  7.30297 

0.50 

0.50 

1.17 

0.22 

1.47 

6.70 

Sam  Var 

2.13  53.3333 

0.25 

0.25 

1.36 

0.05 

2.16 

44.89 

Kurtosis 

-0.35  43.3492 

-151 

-1.91 

-1.30 

-1.47 

-1.46 

0.51 

Skew 

0.54  0.53995 

-0.53 

0.53 

0.43 

0.08  -0.42 

0.79 

Age- 


Group 

2-2 

Stimuli 
Age  Int 

%  Cor 
Det 

Prod  Arous 

Vig 

Dec 

Vig 

Arous- 
Vig  Int 

Mean 

28.40 

1704 

0.73 

0.20 

4.60 

0.22 

2.40 

11.40 

Std  Err 

0.29  1.73699 

0.12 

0.11 

0.19 

0.02 

0.24 

1.22 

Median 

28.00 

168 

1.00 

0.00 

5.00 

0.25 

3.00 

15.00 

Mode 

28.00 

168 

1.00 

0.00 

5.00 

0.25 

3.00 

15.00 

Std  Dev 

1.12  6.72734 

046 

0.41 

0.74 

0.09 

0.91 

4.72 

Sam  Var 

1.26  45.2571 

0.21 

0.17 

0.54 

0.01 

0.83 

22.26 

Kurtosis 

0.38  0.37826 

-0.73 

0.90 

1.32 

2.04 

2.36 

0.98 

Skew 

0.11  0.11226 

-1.18 

1.67 

-1.63 

-1.43 

-1.63 

-1.24 

Age- 

-Group 

Stimuli 

%  Cor 

Vig 

Arous- 

2-3 

Age  Int 

Det 

Prod  Arous 

Dec 

Vig 

Vig  Int 

Mean 

27.80  194.634 

0.78 

0.20 

5.27 

0.46 

5.59 

29.56 

Std  Err 

0.20  1.38756 

0.07 

0.06 

0.11 

0.02 

0.35 

1.99 

Median 

28.00  196 

1.00 

0.00 

5.00 

0.50 

7.00 

30.00 
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Mode 

28.00  196 

1.00 

0.00 

5.00 

0.50  3.00 

15.00 

StdOev 

1.27  8.88469 

0.42 

0.40 

0.71 

0.16  2.27 

12.72 

Sam  Var 

1.61  78.9378 

0.18 

0.16 

0.50 

0.03  5.15  161.85 

Kurtosis 

-0.90  -0.8963 

-0.0? 

0.58 

-0.87 

0.30  -1.07 

-1.08 

Skew 

0.23  0.23169 

-1.41 

1.60 

-0.44 

-0.46  -0.55 

-0.25 

Group 

2-4 

Age- 
Stimuli 
Age  Int 

%  Cor 
Det 

Prod  Arous 

Vig 

Dec 

Vig 

Arous- 
Vig  Int 

Mean 

28.70 . 

229.6 

0.75 

0.23 

6.10 

0.48 

6.18 

37.75 

StdErr 

0.46  3.71497 

0.07 

0.07 

0.11 

0.02 

0.33 

2.20 

Median 

28.50 

228 

1.00 

0.00 

6.00 

0.50 

6.00 

35.00 

Mode 

32.00 

256 

1.00 

0.00 

6.00 

0.42 

5.00 

42.00 

Sid  Dev 

2.94  23.4956 

0=44 

0.42 

0.67 

0.10 

2.09 

13.90 

Sam  Var 

8.63  552.041 

0.19 

0.18 

0.45 

0.01 

4.35  193.12 

Kurtosis 

-1.84  -1.8428 

-0.59 

-0.14 

-0.68 

11.74 

0.49 

0.10 

Skew 

-0.08  -0.079 

-1.2Q 

1.37 

-0.12 

-2.57  -0.44 

-0.07 

Age- 


Stimuli 

%Gor 

Vig 

Arous- 

Groupl 

Age- 

Int 

Det-  iProd  Arous 

Dec 

Vig 

Vig  Int 

Mean 

-42.65  114.477 

0.50 

0.44 

4.45 

0.23 

2.48 

11.09 

StdErr 

0.65 

5.261 

0.05 

0.05 

0.14 

0.01 

0.13 

0.68 

Median 

42.00 

126 

0.50  i 

0.00 

5.00 

0.25 

3.00 

10.00 

Mode 

4o;oo 

153- 

0.00 

0.00 

6.00 

0.25 

3.00 

18.00 

Sid  Dev 

8.0649.3526 

aso 

0.50 

1.30 

0.11 

1.23 

6.35 

SarrfVar 

36.67  2435.68 

0.25 

0.25 

1.70 

0.01 

1.52 

40.31 

Kurtosis 

-0.10 

-1.1722 

-205 

-1.99 

-0.99 

-0.50  -0.06 

-0.70 

Skew 

0.30 

-0.2488 

0.00 

0.23 

-0.39 

-0.34 

0.07 

0.20 

Age- 

Stimuli 

%  Cor 

Vis 

Arous- 

Group  2  -Age 

int 

Pef 

Prod  Arous 

Vig 

Vig  Int 

ty|ean 

27.80  191,581 

0.74 

0-24 

5.27 

0.42 

5.00 

27.26 

Std-Err 

0.21  3.59416 

0.04 

0.04 

0.10 

0.02 

0.22 

1.42 

Medi^fi 

?8.Q0 

196 

100 

0.00 

5.00 

0.42 

5.00 

25.00 

Mode 

27.00 

196 

4.00 

0.00 

6.00 

0.50 

3.00 

15.00 

Std  Dev 

2.26  38.8767 

0.44 

0.43 

1.06 

0.17 

2.37 

15.33 

Sam  Var 

5.12 

1511.4 

0.20 

0.18 

1.13 

0.03 

5.60  234.88 

Kurtosis 

-0.75 

-0.5152 

-0.85 

-0.48 

-0.16 

-0.22  -0.96 

-0.73 

Skew 

0.35 

-0.1556 

-1.08 

1.24 

-0.53 

-0.49  -0.02 

0.36 
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